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EXECUTIVE  SUMMARY 


INTRODUCTION 

The  United  States  Coast  Guard  (USCG)  Research  and  Development  Center  (R&DC)  is 
investigating  the  application  of  advanced  training  technology  to  improve  the  cost-effectiveness  of 
training.  A  major  part  of  this  program  is  the  evaluation  of  Computer-Based  Training  (CBT),  as 
an  alternative  to  traditional  training  approaches.  The  Training  Technologies  Pilot  Study  project 
has  embarked  on  a  study  to  evaluate  the  cost/effectiveness  of  nonresident  CBT,  in  comparison 
with  instructor-led  resident  training  at  a  USCG  training  center.  The  one-week,  instructor-led 
AN/WSC-3(v)7  UHF  Transceiver  Maintenance  course,  offered  at  the  USCG  Training  Center, 
Petaluma,  California  (TRACEN  Petaluma),  was  selected  as  the  baseline  for  this  evaluation.  In 
addition,  a  duty  station  implementation  analysis  was  performed  to  identify  issues  and  recommend 
solutions  based  on  the  experiences  of  the  various  USCG  units  (duty  stations)  whose  members 
participated  in  the  nonresident  CBT  portion  of  the  study.  Many  of  the  issues  identified,  and  the 
recommendations  proposed,  would  be  common  to  other  nonresident  media  considerations  (e.g., 
interactive  video-teletraining  (IVT),  correspondence).  Therefore,  references  to  nonresident 
training  are  defined  as  the  entire  range  of  media  that  may  be  employed  by  the  USCG  to  deliver 
training  to  personnel  at  their  duty  stations. 

METHODOLOGY 

A  CBT  version  of  the  resident  ANAVSC-3  course  was  developed  for  the  study  by  an  independent 
contractor.  The  training  effectiveness  evaluation  was  based  on  an  experiment  comparing  three 
training  strategies,  using  three  groups  of  students  from  units  around  the  country: 

•  ANAVSC-3  CBT  course  conducted  at  duty  stations  (CBT  Duty  Station  Group)  -  The  CBT 
course  was  shipped  to  each  student’s  duty  station,  prepackaged  on  a  multimedia  laptop 
computer.  A  facilitator  at  each  duty  station  coordinated  the  training.  The  students  were 
tested  at  TRACEN  Petaluma  after  course  completion. 

•  ANAVSC-3  CBT  course  conducted  at  TRACEN  Petaluma  (CBT  Remote  Group)  -  These 
students  received  the  same  CBT  course  as  the  duty  station  students,  but  in  the  school 
environment  at  TRACEN  Petaluma.  They  were  tested  at  TRACEN  Petaluma  after  course 
completion. 

•  Traditional,  instructor-led  ANAVSC-3  resident  course  conducted  at  TRACEN  Petaluma 
(Resident  Group)  -  These  students  received  the  traditional  one-week  course.  They  were 
tested  at  TRACEN  Petaluma  after  course  completion. 

Training  effectiveness  was  evaluated  at  TRACEN  Petaluma,  on  the  basis  of:  1)  pre-  and  post¬ 
training  knowledge  tests;  2)  operation  and  maintenance  performance  measured  by  evaluators 
during  troubleshooting  of  malfunctioning  AN/WSC-3(v)7  UHF  transceivers;  and  3)  students’ 
reactions  given  in  modified  C-School  Critiques.  Training  time  was  also  assessed. 

Staff  at  about  half  of  the  participating  duty  stations  were  interviewed  after  completion  of  all 
training  and  testing  activities.  The  purpose  was  to  identify  issues  and  potential  problems 
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associated  with  the  implementation  of  training  at  duty  stations,  including  suggestions  for  dealing 
with  them. 

CONCLUSIONS 

The  computer-based  training  course  for  AN/WSC-3  operation  and  maintenance  was  successfully 
conducted  at  13  USCG  duty  stations  in  different  parts  of  the  United  States.  This  training  was 
equal  in  effectiveness  to  training  conducted  at  a  USCG  training  center,  including  both  the  CBT 
course  and  the  traditional  instructor-led  resident  course.  Nonresident  operation  and  maintenance 
training  conducted  at  USCG  duty  stations,  therefore,  can  be  as  effective  as  courses  conducted  at 
training  centers. 

Substantially  reduced  training  time  is  a  potentially  major  benefit  of  CBT,  in  comparison  with 
instructor-led  resident  training.  The  nonresident  AN/WSC-3  CBT  course  required  an  average 
70%  less  training  time  than  the  equivalent  instructor-led  resident  course.  Similar  reductions  in 
training  time  were  achieved  with  the  CBT  course  conducted  at  the  training  center. 

The  students,  commanders  and  other  staff  interviewed  at  the  participating  duty  stations  generally 
support  the  implementation  of  nonresident  training  (e.g.,  CBT,  IVT).  They  feel  that  nonresident 
training  can  succeed  in  the  USCG,  and  will  be  a  major  benefit  to  duty  stations.  Nonresident 
training  is  viewed  by  duty  station  staff  as: 

•  Having  less  adverse  impact  than  resident  training  on  the  overall  team’s  ability  to  perform 
its  mission;  resulting  in  greater  staff  availability;  minimizing  unit  disruptions;  easing  unit 
staffing  problems;  and  resulting  in  increased  training  efficiency. 

•  Assisting  their  general  unit  training  requirements. 

•  A  partial  solution  to  problems  associated  with  minimal-crewed  vessels. 

•  Helping  the  pipeline  training  problem. 

Duty  station  staff  believe  existing  obstacles  must  be  overcome  with  careful  organization,  and 
development  and  execution  of  the  training  process.  Important  issues  identified  were: 

•  The  training  process  and  procedures  must  provide  the  unit  flexibility  to  schedule  and 
conduct  training  in  accordance  with  the  unit’s  particular  situation.  This  includes  a  strong 
training  management  organizational  structure  at  the  duty  station,  and  at  other  levels  in  the 
USCG. 

•  Sufficient  time  should  be  provided  daily,  during  normal  work  hours,  for  the  conduct  of 
training.  Training  should  be  scheduled  like  any  other  work  task. 

•  Interruptions  must  not  be  allowed  while  students  are  actively  engaged  in  training,  except 
for  real  emergencies. 

•  The  area  and  room  in  which  a  student  is  training  must  have  an  environment  conducive  to 
learning,  such  as  minimizing  distractions. 

•  Effective  approaches  must  be  developed  to  achieve  hands-on  training  and  student 
certification  as  part  of  the  duty  station  training  process. 
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RECOMMENDATIONS 


The  USCG  should  proceed  with  the  implementation  of  nonresident  CBT  at  duty  stations.  This 
will  require  selection  of  courses  to  convert,  conversion  of  the  courses,  and  establishment  of  an 
effective  training  process  at  duty  stations. 

The  many  issues  associated  with  large-scale  implementation  of  nonresident  CBT,  and  other 
nonresident  media  delivery  methods,  are  expected  to  require  some  re-organization  of  the  USCG’s 
infrastructure,  affecting  both  operations  and  training.  These  should  include  organizational 
changes  for  managing  resident  and  nonresident  training,  and  the  necessary  checks-and-balances 
to  assure  that  effective  training  is  regularly  achieved  at  duty  stations. 

Special  emphasis  should  be  placed  on  assuring  that  the  initial  courses  converted  to  a  nonresident 
mode  (e.g.,  CBT,  IVT)  are  as  effective  as  the  resident  courses  they  replace. 

Many  suggestions  for  establishment  of  an  effective  training  process  at  duty  stations  were 
received  during  the  interviews,  and  are  addressed  in  the  report.  Examples  are: 

•  Strong  overt  command  support  for  duty  station  training,  at  all  levels  across  the  USCG, 
must  be  given  and  demonstrated. 

•  A  student-help  hotline  is  recommended  to  provide  all  duty  station  students  with 
immediate  access  to  a  subject  matter  expert. 

•  A  section  should  be  added  to  the  USCG’s  PCO/PXO  Course,  addressing  how  to  establish 
and  conduct  an  effective  training  process  at  duty  stations. 

•  Software  tools  should  be  developed  for  the  duty  station,  to  support  training  management, 
monitoring  and  oversight  of  student  training,  and  administrative  coordination. 

An  internal  USCG  marketing  effort  should  be  established,  to  inform  personnel  at  all  levels  about 
nonresident  training  (e.g.,  CBT,  IVT). 

Guidelines  should  be  developed  for  all  levels  of  the  USCG  (e.g.,  students,  command  chain) 
addressing  how  to  develop  and  conduct  an  effective  training  process. 

A  centralized  quality  assurance  process  should  be  developed  as  a  check-and-balance,  to  assure 
that  an  adequate  level  of  training  effectiveness  is  developed  and  maintained  at  all  duty  stations. 
This  process  should  also  be  responsible  for  maintaining,  modifying  and  upgrading  courses  as 
necessary. 

A  detailed  transition  plan  is  required  to  guide  the  implementation  of  nonresident  training  (e.g., 
CBT,  IVT)  across  the  USCG.  This  plan  will  require  research  to  resolve  many  of  the  identified 
issues  affecting  the  establishment  of  an  effective  training  process  at  duty  stations. 
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INTRODUCTION 


The  United  States  Coast  Guard  (USCG)  conducts  many  training  courses  to  develop  and  improve 
the  proficiency  of  its  personnel.  Although  a  variety  of  training  methods  and  media  are  used, 
most  of  the  training  uses  traditional  instructor-led  methods  conducted  at  training  centers.  The 
rapid  growth  of  modem  training  technology,  much  of  which  is  computer-based,  provides 
opportunities  for  effectiveness  and  efficiency  improvements  in  the  delivery  of  training  across  the 
USCG.  This  project  evaluated  the  potential  effectiveness  of  nonresident  computer-based  training 
(CBT),  conducted  at  USCG  duty  stations.  In  addition,  a  duty  station  implementation  analysis 
was  performed  to  identity  issues  and  recommend  solutions  based  on  the  experiences  of  the 
operational  units  (duty  stations)  whose  members  participated  in  the  nonresident  CBT  portion  of 
the  study.  Many  of  the  issues  identified,  and  the  recommendations  proposed,  would  be  common 
to  other  nonresident  media  considerations  (e.g.,  interactive  video-teletraining  (IVT)).  Therefore, 
references  to  nonresident  training  are  defined  as  the  entire  range  of  media  that  may  be  employed 
by  the  USCG  to  deliver  training  to  personnel  at  their  duty  stations.  This  document  reports  the 
methodology  and  findings  of  the  research. 

Note,  this  project  is  one  of  several  phases  accomplished  as  part  of  the  Training  Technologies 
Pilot  Study.  A  separate  cost  analysis  of  nonresident  CBT,  in  comparison  with  traditional  resident 
training,  was  performed  as  another  part  of  the  Training  Technologies  Pilot  Study  (Kingsley, 
Cummings,  and  Hammell,  1998). 

1.1  BACKGROUND 

1.1.1  Advanced  Training  Technology 

The  past  forty  years  have  seen  tremendous  advancements  in  training  technology,  and  its 
economical  availability  (Office  of  Technology  Assessment,  1988;  Reisman  and  Carr,  1991). 

CBT,  in  various  forms,  has  been  in  use  for  well  over  30  years.  Its  relatively  high  cost  in 
comparison  with  traditional  training  methods,  however,  has  limited  the  use  of  CBT  until  more 
recent  years.  CBT  has  characteristics  that  can  be  used  to  enhance  the  training  process,  such  as  its 
ease  in  providing  for  student  interactivity,  presentation  of  graphics  and  animation,  and  allowance 
for  self-paced  instruction.  For  example,  student  interactivity  (Campbell,  1988),  visual  media 
presentation  (Shriver,  Shriver  and  Bunderson,  1984;  Shulman,  Vogel  and  Reilly,  1986),  and  self- 
paced  instruction  (Hamel  and  Clark,  1986)  have  been  shown  to  facilitate  the  learning  of  complex 
tasks.  These  types  of  student  visualization  and  interactive  processes  require  substantial 
computing  power.  Until  recently,  all  but  the  most  basic  CBT  training  applications  required 
expensive  workstations  to  achieve  the  media  characteristics  necessary  to  support  complex 
training.  In  addition,  the  development  of  the  sophisticated  training  software  and  the  operation  of 
networked  CBT  systems  were  also  expensive.  Consequently,  computers  were  primarily  used  in 
training  to  drive  sophisticated  simulators,  where  their  high  cost  could  be  justified  in  support  of 
operational  team  training.  In  essence,  the  simulator’s  development  and  operational  costs  were 
justified  in  comparison  with  the  cost,  and  safety,  of  operating  ships  and  other  equipment. 
Although  educators  were  well  aware  of  the  potential  of  CBT  in  the  classroom,  computers  were 
seldom  used  in  this  context  until  recent  years. 
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Computer-based  training,  which  began  in  the  late  1950’s  (Reisman  et.  al.,  1991),  began 
infiltrating  traditional  training  environments  in  the  early  to  mid  1980’s,  ostensibly  as  interactive 
video  disk  (Wright,  1993),  and  then  later  as  multimedia.  Studies  have  touted  the  potential 
benefits  of  the  various  forms  of  CBT.  For  example,  studies  have  generally  found  a  reduction  in 
training  time  from  CBT  in  comparison  with  the  traditional  training,  with  an  average  of  50% 
reduction  estimated  (Miller,  1990).  This  has  been  attributed  to  several  factors,  including  self- 
paced  training,  personalized  instruction  (e.g.,  beginner-level  instruction  versus  advanced-level 
instruction)  student  interactivity  and  consistent  training. 

The  increasing  capabilities  and  lowering  cost  of  the  microcomputer,  together  with  the  evolving 
development  of  sophisticated  software  tools  to  construct  and  manage  CBT  media,  have 
substantially  reduced  the  cost  of  CBT  courses.  Today,  CBT  can  be  cost-competitive  with 
traditional  training  methods.  The  cost  trade-off  involves  hardware,  software  development, 
training  content  development,  instructor  and  student  time,  laboratory  hardware  and  operation, 
course  distribution  and  management,  and  student  travel. 

Today,  CBT  may  be  delivered  in  several  forms,  such  as  interactive  video  disk  and  interactive 
courseware  (ICW).  The  multimedia  CBT  trend  is  to  make  a  variety  of  computer-based  media 
available  on  a  low-cost  desktop  or  laptop  personal  computer,  such  that  a  training  program  can 
use  the  best  mix  of  media  to  deliver  the  course. 

1.1.2  Coast  Guard  Training  Issues 

Traditional,  instructor-led  training  methods  used  at  Coast  Guard  Training  Centers  are  effective, 
having  been  evolved  over  many  years.  However,  two  factors  suggest  the  need  to  explore  changes 
to  these  traditional  methods.  Those  factors  are  the  continual  tightening  of  annual  budgets  which 
requires  a  constant  vigilance  for  ways  of  reducing  costs,  and  the  current  training  system  is 
somewhat  inflexible  in  meeting  the  range  of  training  needs  of  USCG  units.  The  availability  of 
effective  modem  training  technologies  presents  a  potential  opportunity  to  reduce  training  costs, 
and  increase  the  responsiveness  of  USCG  training  programs,  while  maintaining  training 
effectiveness.  In  essence,  this  is  a  technology  push. 

Each  instructor-led  resident  course  has  a  limit  on  the  number  of  students  who  can  attend  a  class; 
often  resulting  in  a  demand  for  student  positions  that  exceeds  the  space  available.  Consequently, 
USCG  units  have  a  limit  on  the  number  of  personnel  who  can  be  trained  in  a  particular  specialty 
(i.e.,  training  quotas),  and  are  unable  to  get  their  training  needs  met  at  the  appropriate  time. 
Furthermore,  resident  training  center  courses  are  usually  scheduled  at  fixed  times  during  the  year. 
These  times  may  not  be  convenient  to  the  work  demands  of  particular  units,  with  the  result  that 
personnel  may  not  receive  the  desired  training  even  if  they  can  obtain  a  student  position  for  the 
course.  Finally,  personnel  in  many  cases  must  travel  great  distances  from  their  units  to  attend  a 
resident  course.  In  some  cases  these  classes  may  be  several  weeks  in  length.  During  the  training 
period,  attending  personnel  would  not  be  available  to  assist  their  units  should  an  emergency 
occur.  The  inherent  inflexibility  of  the  current  training  system  places  a  burden  on  USCG  units 
and  personnel. 

Costs  to  operate  and  staff  a  resident  training  center  are  substantial.  The  potential  exists,  through 
the  use  of  CBT  and  other  modem  training  technologies  (e.g.,  interactive  video-teletraining),  to 
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reduce  training  costs  by  shifting  a  portion  of  training  from  the  resident  training  centers  to  the 
USCG  units.  This  may  be  achievable  with  a  substantial  net  cost  savings,  through  the  application 
of  modem  training  technologies.  This  shift  of  training  locale,  coupled  with  the  use  of  modem 
training  technologies,  would  provide  additional  benefits  to  USCG  units,  such  as  vastly  increased 
training  flexibility. 

1.2  PROBLEM  AND  OBJECTIVE 

As  a  result  of  the  training  issues  stated,  the  U.S.  Coast  Guard  Headquarters’  Office  of  Training 
and  Performance  Consulting  (G-WTT)  tasked  the  R&DC  to  perform  a  study  of  modem  training 
technologies.  This  study  is  referred  to  as  the  “Training  Technologies  Pilot  Study.”  The  goal  of 
the  study  was  to  determine  the  potential  impact  of  using  state-of-the-industry  (modem)  training 
technologies  as  alternative  methods  for  meeting  USCG  training  needs.  A  major  advantage  of 
CBT  is  to  have  a  substantial  amount  of  training  done  at  duty  stations,  therefore  the  effectiveness 
of  nonresident  training  becomes  an  important  issue.  The  problem  to  be  addressed  was,  can 
nonresident  training  at  duty  stations  be  as  effective  as  instructor-led  training  at  resident  training 
centers.  This  issue  does  not  necessarily  pertain  to  all  resident  training,  but  rather  to  the  training 
of  skills  and  knowledge  that  should  be  responsive  to  CBT  methods. 

The  objective  of  this  study  was  to  determine  the  feasibility  of  conducting  training  at  duty 
stations,  in  lieu  of  training  centers.  During  an  earlier  phase  of  the  project  the  USCG  selected  the 
operation  and  maintenance  course  for  the  AN/WSC-3(v)7  UHF  transceiver  as  the  context  of  the 
empirical  evaluation.  This  one-week  course  addressed  skills  and  knowledge  that  appeared 
conducive  to  the  CBT  medium  and  delivery  method.  The  ANAVSC-3  resident  course  was 
developed  into  CBT  form  for  training  at  duty  stations.  The  objective  of  this  study,  more 
specifically  therefore,  was  to  empirically  determine  if  the  nonresident  ANAVSC-3  CBT  course, 
to  be  conducted  at  duty  stations,  could  be  as  effective  as  the  instructor-led  resident  course  taught 
at  a  Coast  Guard  training  center.  An  outgrowth  of  this  investigation  would  be  the  demonstration 
of  nonresident  training  at  duty  stations. 

A  second  objective  was  to  identify  important  issues  and  problems  associated  with  the 
implementation  of  nonresident  training  at  duty  stations.  This  was  accomplished  by  interviewing 
unit  staff  where  personnel  took  the  nonresident  CBT  version  of  the  ANAVSC-3  course  at  their 
duty  station  (unit).  Based  on  the  results  of  those  interviews,  recommended  solutions  are 
presented  in  this  report. 

1.3  REPORT  ORGANIZATION 

The  main  body  of  the  report  is  organized  into  the  following  sections: 
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2.  Methodology 

3.  Training  Effectiveness  Results 

4.  Training  Effectiveness  Discussion 

5.  Duty  Station  Implementation  Analysis  Findings 

6.  Conclusions 

7.  Recommendations 

2  METHODOLOGY 

The  methodology  used  during  the  study  addressed  the  1)  training  effectiveness  evaluation,  and  2) 
analysis  of  implementation  issues.  The  training  effectiveness  evaluation  addressed  the  design 
and  conduct  of  an  experiment  to  evaluate  the  training  effectiveness  of  a  nonresident  ANAVSC-3 
CBT  course  conducted  at  duty  stations,  in  comparison  with  traditional  instructor-led  courses 
conducted  at  a  resident  training  center.  The  analysis  of  implementation  issues  addressed  the 
conduct  of  interviews  at  a  subset  of  the  duty  stations  participating  in  the  nonresident  CBT,  to 
identify  issues  and  problems  associated  with  duty  station  training,  and  to  recommend  actions  for 
dealing  with  them. 

2.1  TRAINING  EFFECTIVENESS  EVALUATION  METHODOLOGY 


2.1.1  Overview 


The  evaluation  of  nonresident  training  focused  on  a  computer-based  training  course  conducted  at 
USCG  duty  stations  across  the  country,  in  place  of  a  traditional  instructor-led  course  conducted 
at  a  resident  training  center.  After  careful  consideration  of  several  factors,  the  AN/WSC(v)7 
UHF  Transceiver  Maintenance  Course  (ANAVSC-3  course)  was  selected  by  the  USCG  to  serve 
as  the  comparative  baseline  for  the  study.  Following  course  selection,  a  CBT  version  of  the 
selected  course  was  developed  to  be  equivalent  to  the  instructor-led  resident  course.  Delivery  of 
the  CBT  version  of  the  course  would  be  on  multimedia  laptop  computers  at  duty  stations. 

A  formal  experiment  was  conducted  to  compare  the  effectiveness  of  three  training  approaches, 
with  three  different  groups  of  students: 

1 .  The  ANAVSC-3  CBT  course  conducted  at  the  students’  respective  duty  stations.  This  group 
is  called  the  CBT  Duty  Station  Group  (DS). 

2.  The  ANAVSC-3  CBT  course  conducted  at  the  USCG  Training  Center  in  Petaluma,  California 
(TRACEN  Petaluma).  This  group  is  called  the  CBT  Remote  Group  (RM). 

3.  The  traditional  instructor-led  ANAVSC-3  resident  course  conducted  at  TRACEN  Petaluma. 
This  group  is  called  the  Resident  Group  (RS). 

The  CBT  course  was  delivered  to  the  CBT  Duty  Station  Group  students  prepackaged  on  a 
multimedia  laptop  computer,  shipped  to  each  student’s  duty  station.  A  facilitator,  designated  by 
the  commanding  officer  at  each  duty  station,  was  charged  with  overseeing  the  conduct  of  the 
training,  and  the  data  collected  at  the  duty  station.  Upon  completion  of  the  CBT  course,  the 
students  were  sent  on  Temporarily  Assigned  Duty  (TAD)  to  TRACEN  Petaluma  to  complete  the 
remaining  tasks  in  the  experiment. 
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The  CBT  Remote  Group  students  received  the  prepackaged  CBT  course  in  the  schoolhouse 
environment  at  TRACEN  Petaluma.  A  facilitator  coordinated  the  training,  following  a  process 
similar  to  that  required  at  the  duty  stations.  Only  the  training  setting  (i.e.,  TRACEN  Petaluma 
vice  a  duty  station)  differed  between  the  CBT  Remote  Group  and  the  CBT  Duty  Station  Group. 
All  tasks  related  to  the  experiment  were  completed  at  TRACEN  Petaluma. 

The  Resident  Group  students  received  the  traditional  one-week  instructor-led  AN/WSC-3  course, 
delivered  at  TRACEN  Petaluma.  All  tasks  related  to  the  experiment  were  completed  at 
TRACEN  Petaluma. 

The  effectiveness  evaluation  was  designed  in  accordance  with  the  4-level  model  of  Kirkpatrick 
(1974)  and  others,  with  focus  on  assessment  at  Levels  1  and  2.  The  students’  reaction  to  the 
training  formed  the  Level  1  evaluation,  which  was  assessed  using  a  questionnaire.  The  Level  2 
evaluation  addressed  the  students’  mastery  of  the  course,  assessing  their  skills  and  knowledge 
during  troubleshooting  on  actual  AN/WSC-3  UHF  transceivers.  This  was  accomplished  with  a 
knowledge  test,  and  with  troubleshooting  of  actual  malfunctioning  AN/WSC-3(v)7  UHF 
transceivers.  It  is  important  to  note  that  troubleshooting  of  actual  malfunctioning  AN/WSC- 
3(v)7  UHF  transceivers  for  all  three  groups  was  performed  at  TRACEN  Petaluma  for  purposes  of 
experimental  control  and  logistics.  These  levels  provided  a  cumulative  evaluation  of  the  CBT 
course,  relative  to  the  resident  course. 

Evaluation  of  the  transfer  of  training  from  the  course  to  on-the-job  performance  (Level  3 
evaluation)  was  not  performed.  The  Level  4  evaluation  (i.e.,  changes  in  organizational 
performance  resulting  from  the  training)  was  evaluated  in  the  form  of  training  delivery  cost 
(Kingsley,  Cummings,  and  Hammell,  1998). 

2.1.2  AN/WSC-3  Course 

The  AN/WSC-3(v)7  UHF  Transceiver  Maintenance  course  was  one  of  several  courses 
considered  as  the  comparative  baseline  for  the  evaluation.  Members  of  the  USCG  project  team, 
selected  the  AN/WSC-3  course  selection  after  careful  consideration  of  several  factors,  including: 

•  Maintenance  and  operation  course  content. 

•  Representative  of  USCG  operation  and  maintenance  courses. 

•  Reasonable  demand  for  the  course. 

•  One-week  length,  which  was  within  a  feasible  CBT  development  budget. 

•  Course  content  was  amenable  to  the  CBT  medium. 

•  The  current  AN/WSC-3  resident  course  was  considered  a  “good’  course,  and  hence 
provided  a  good  baseline  for  comparison  with  the  CBT  course  version. 

2.1.2.1  Instructor-led  Resident  Course 

The  instructor-led  AN/WSC-3  resident  course  resides  at  TRACEN  Petaluma.  The  resident 
course  convenes  about  12  times  per  year,  with  a  maximum  class  size  of  six  students.  The 
limiting  factors  on  class  size  are  a  combination  of  the  number  of  instructors  available,  the  size  of 
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the  room  where  the  class  takes  place,  and  the  number  of  AN/WSC-3(v)7  UHF  transceivers 
available  at  the  training  center.  The  one-week  resident  course  is  conducted  in  a  laboratory 
environment,  consisting  primarily  of  lecture,  discussion,  and  hands-on  maintenance  problems 
using  the  ANAVSC-3(v)7  UHF  transceivers.  Each  student  is  given  a  handbook,  which  provides 
detailed  guidance  tailored  to  the  operation  and  maintenance  procedures  used  by  the  USCG  for 
this  equipment.  A  set  of  U.S.  Navy  published  AN/WSC-3  technical  manuals  is  also  used  during 
the  training.  Successful  completion  of  the  course  is  based  on  interaction  with  the  instructor 
during  the  class.  Upon  completion  of  the  course  the  student  is  given  a  certificate  designating  that 
the  student  has  successfully  completed  the  course.  Once  a  student  returns  to  the  duty  station,  the 
unit’s  personnel  office  processes  paperwork  (i.e.,  Career  Development  Worksheet)  which  will 
assign  a  corresponding  qualification  code  in  the  student’s  personnel  record.  In  addition  to 
receiving  the  certificate  upon  completion  of  the  course,  students  complete  a  standard  course 
critique,  which  provides  data  regarding  student  reaction  to  the  course.  The  AN/WSC-3  resident 
course  served  as  the  yardstick  (baseline)  against  which  the  CBT  version  of  the  course  was 
compared. 


2.1.2.2  CBT  Course 


The  AN/WSC-3  CBT  course  was  developed  as  interactive  courseware  (ICW)  by  Analysis  and 
Technology,  Inc.,  with  the  subject-matter-expert  (SME)  assistance  of  AN/WSC-3  course 
instructors  from  TRACEN  Petaluma.  For  economic  and  investigative  reasons,  the  development 
goal  was  to  achieve  a  CBT  course  that  was  equivalent  to  the  instructor-led  resident  course  (i.e., 
not  superior  to  it).  The  resulting  CBT  course  was  developed  for  delivery  on  a  multimedia  laptop 
computer.  The  CBT  course  was  divided  into  seven  modules.  The  first  five  modules  contained 
instruction  on  the  maintenance  and  troubleshooting  of  AN/WSC-3(v)7  UHF  transceivers.  The 
sixth  module  contained  a  series  of  troubleshooting  practice  problems,  and  the  seventh  module 
tested  student  troubleshooting  performance  (these  test  scores  were  not  evaluated  in  performing 
the  effectiveness  evaluation  analysis).  Student  study  times  were  accessible  from  computer  data 
saved  during  each  training  session. 

A  mixture  of  tutorial  and  problem-solving  instructional  strategies  was  used,  which  promoted 
active  student  involvement  during  the  training  process.  Extensive  graphics  were  presented, 
including  diagrams,  views  of  the  AN/WSC-3  unit  and  its  internal  components,  an  attention- 
getting  humorous  old  Electronics  Technician  Chief  to  occasionally  emphasize  certain  points,  and 
an  introductory  video.  Much  of  the  instructional  presentation  used  both  visual  and  audio  modes 
of  presentation,  with  headphones  provided  to  minimize  interference  with  others  in  the  work 
environment.  The  student-courseware  interface  was  highly  refined,  and  representative  of  the 
finest  CBT  programs  available  today.  The  tutorial  directed  the  student  in  a  logical  progression 
through  each  instruction  topic  and  set  of  problems,  and  allowed  the  student  to  review  any  topic 
on-demand.  The  students  listened  to  presented  information  while  watching  the  graphics, 
answered  multiple  choice  questions,  configured  and  operated  a  simulated  AN/WSC-3(v)7  UHF 
transceiver,  located  and  removed  components,  connected  external  test  equipment,  and  ran  tests. 

It  was  anticipated  that  students  would  require  between  six  and  fourteen  hours  to  complete  this 
self-paced  course. 

An  additional  component  of  the  CBT  course  was  a  two-hour  hands-on  training  segment,  to 
facilitate  student  transfer  of  skills  and  knowledge  from  the  CBT  medium  to  the  actual 
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AN/WSC-3(v)7  UHF  transceiver.  Since  the  objective  of  this  investigation  was  to  evaluate  the 
potential  of  nonresident  training,  rather  than  evaluate  the  strengths  and  weaknesses  of  CBT,  it 
was  necessary  to  achieve  a  CBT  course  configuration  similar  to  everyday  training  (i.e.,  typical 
production  training,  rather  than  a  one-time  experimental  configuration).  Therefore,  the  hands-on 
training  segment  was  performed  following  the  completion  of  the  CBT  portion  of  the  AN/WSC-3 
course.  For  purposes  of  the  study,  which  included  experimental  control,  the  hands-on  training 
segment  was  performed  at  TRACEN  Petaluma  as  part  of  the  students’  lab  familiarization.  It 
should  be  noted  that  during  normal  training  activities,  any  hands-on  training  required  as  part  of  a 
nonresident  course  would  likely  be  accomplished  at  or  near  the  student’s  duty  station. 

2.1.3  Experiment  Design 

A  formal  experiment  was  conducted  to  evaluate  the  effectiveness  of  the  AN/WSC-3  CBT  course 
at  duty  stations.  The  evaluation  was  based  on  comparative  performance  between  three  different 
AN/WSC-3  training-delivery  treatments,  each  of  which  was  administered  to  a  separate  group  of 
ET  students: 

CBT  Duty  Station  (DS):  AN/WSC-3  CBT  course  taught  at  each  student’s  current  duty 

station. 

CBT  Remote  (RM):  AN/WSC-3  CBT  course  taught  at  TRACEN  Petaluma. 

Resident  (RS):  Existing  40-hour  AN/WSC-3  course  taught  at  TRACEN 

Petaluma. 

2.1.3.1  Independent  Variable 

The  three  training  methods  (i.e.,  treatment  conditions)  comprised  the  independent  variable  of  the 
experiment.  The  comparison  between  DS  and  RS  groups  provided  the  evaluation  of  the  DS 
training  effectiveness,  with  the  RS  group  acting  as  the  control.  Comparisons  between  these 
groups  and  the  RM  group  provided  evaluation  of  the  CBT  course’s  effectiveness  when  the 
potential  distractions  of  the  duty  station  environment  were  removed  (i.e.,  CBT  in  a  training 
center  environment). 


2.1. 3.2  Dependent  Variables  (Performance  Measures) 

Several  measures  of  performance  were  used  as  the  basis  of  student  and  group  performance, 
comprising  the  dependent  variables,  as  follows: 

Posttest  -  Paper-and-pencil  test.  This  test,  used  to  assess  the  students’  knowledge  after 
completion  of  the  AN/WSC-3  course,  was  administered  to  all  students  at  TRACEN 
Petaluma  before  the  start  of  other  testing  activities.  Differences  between  Pretest  and 
Posttest  scores  were  also  planned  to  provide  a  measure  of  training  gain  resulting  from  the 
course  (Note,  the  Pretest  is  addressed  in  the  following  subsection). 

Hands-on  Problems  -  A  practical  test  in  which  each  student  had  to  troubleshoot  and 
repair  malfunctioning  AN/WSC-3(v)7  UHF  transceivers.  The  five  separate  problems 
were  similar  to  those  that  an  Electronic  Technician  might  experience  with  a  AN/WSC- 
3(v)7  UHF  transceiver  at  the  duty  station.  Each  student’s  proficiency  was  evaluated 
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while  solving  each  problem.  Student  performance  on  these  problems  was  considered  a 
reasonable  facsimile  of  their  likely  performance  in  similar  situations  at  a  duty  station,  and 
thus  a  bottom-line  indicator  of  training  effectiveness. 

Problem  Time  -  The  amount  of  time  the  student  spent  solving  each  hands-on  problem. 

Training  Time  -  The  amount  of  time  the  student  spent  doing  AN/WSC-3  training 
activities. 


2.1.3.3  Additional  Student  Data 


Additional  data  were  collected,  as  follows: 

Pretest  -  Paper  and  pencil  test,  equivalent  to  the  Posttest.  It  was  administered  to  all 
students,  to  assess  their  knowledge  prior  to  starting  the  AN/WSC-3  training,  and  to 
control  for  possible  imbalance  in  the  student  groups. 

C-School  Critique  -  Modified  versions  of  the  Coast  Guard’s  standard  C-School  Critique 
(CGCSCH-5),  tailored  for  each  group.  The  Critique  was  completed  by  all  students 
following  testing. 

Background  Questionnaire  -  Detailed  information  pertaining  to  each  student’s 
education  and  experience,  collected  using  a  questionnaire  format. 

The  performance  measures  and  the  C-School  Critique  are  explained  more  fully  in  later 
subsections. 

It  should  be  noted  that  data  were  collected  for  a  large  number  of  parameters,  with  many  statistical 
tests  performed.  The  statistical  techniques  that  generate  estimates  of  significance  assume  a 
single  test  is  performed,  rather  than  many  tests  on  different  parameters.  Consequently,  the 
aggregate  set  of  results  has  a  heightened  prospect  of  finding  significant  comparisons.  This 
important  consideration  must  be  taken  into  account  with  the  careful  interpretation  of  results. 

2.1.4  Students 

The  student  sample  consisted  of  Electronic  Technician  (ET)  and  non-rated  electronics  personnel, 
who  were  designated  to  take  the  AN/WSC-3  course.  The  plan  was  to  train  a  total  of  54  students, 
18  students  in  each  group.  Assignment  of  students  to  each  group,  and  to  one  of  the  three  course 
sections  for  each  group,  was  done  in  accordance  with  their  designation  by  the  USCG  Training 
Quota  Center  (TQC)  prior  to  the  convening  of  each  course  section.  This  student  assignment 
process  was  considered  to  be  essentially  random,  for  the  purposes  of  data  analysis. 

A  total  of  46  students  were  trained  across  the  three  groups.  The  students  in  each  group  consisted 
of  ET-1,  ET-2,  ET-3  and  non-rated  personnel.  The  distribution  of  students  across  the  groups, 
and  their  ratings,  are  presented  in  Table  1.  The  differences  between  the  planned  and  actual 
numbers  reflect  scheduling  conflicts.  Interestingly,  the  DS  group  resulted  in  the  largest  number 
of  students. 
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TABLE  1.  DISTRIBUTION  OF  STUDENTS  AND  STUDENT-RATINGS  ACROSS 
GROUPS  _ _ _ _ 


GROUP 

STUDENT  RATING 

TOTAL 
(Per  Group) 

ET1 

ET2 

ET3 

Non-Rated 

DS 

4 

4 

7 

2 

17 

RM 

3 

5 

7 

1 

16 

RS 

3 

1 

7 

2 

13 

Totals 

10 

10 

21 

5 

46 

The  students  in  all  groups  came  from  a  variety  of  operational  units  around  the  country,  and  from 
A-School.  The  duty  stations  at  which  the  CBT  training  was  conducted  were,  similarly,  a 
representative  cross-section.  The  DS  group  students  represented  both  afloat  and  shoreside  units 
(i.e.,  6  and  1 1  students,  respectively),  from  East,  West  and  Gulf  coast  locations. 

2.1.5  Performance  Measurement  Tools 

2.1.5.1  Hands-On  Problems 

A  “hands-on”  test  was  developed  to  assess  student  post-training  proficiency  in  performing 
AN/WSC-3  troubleshooting  and  corrective  maintenance,  working  with  actual 
AN/WSC-3(v)7  UHF  transceivers  in  the  lab  at  TRACEN  Petaluma.  The  selected  five  problems 
presented  realistic  troubleshooting  and  maintenance  challenges  in  a  near-work-like  environment, 
similar  to  that  at  a  duty  station. 

The  student  started  each  problem  with  a  note  of  some  general  symptoms,  and  had  to  do  whatever 
was  required  to  return  that  transceiver  to  service  (For  example,  run  tests,  diagnose/isolate, 
remove  and  replace  components).  Two  senior  ETs,  acting  as  independent  evaluators,  observed 
and  evaluated  each  student's  performance,  according  to  a  structured  set  of  criteria.  The  data  from 
this  set  of  varied  tests  comprised  the  student’s  hands-on  problem  score. 

Each  of  the  five  problems  was  configured  by  inserting  a  faulted  board,  and  making  other  related 
malfunction  changes,  in  an  AN/WSC-3(v)7  UHF  transceiver.  The  five  hands-on  problems, 
identified  by  their  respective  faults,  are  listed  in  Table  2.  (The  symptoms  and  corrective  action 
for  each  problem  are  identified  in  Appendix  A). 

Student  Proficiency  Assessment.  The  two  evaluators  independently  assessed  each  student’s 
performance  during  each  problem,  awarding  component  scores  throughout  each  problem. 

Student  performance  was  assessed  in  accordance  with  pre-determined  performance  criteria,  using 
a  highly  structured  evaluation  technique  developed  for  this  experiment.  The  structure’s 
framework  was  focused  on  six  dimensions  of  performance,  with  task-items  under  each,  which 
mirrored  the  general  troubleshooting  and  maintenance  process  students  were  expected  to  follow: 


Symptoms 

Sectionalization 

Localization 
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Isolation 

Corrective  Action 

Overall  Performance  (Documentation  and  Safety) 


TABLE  2.  LIST  OF  HANDS-ON  PROBLEMS 


Problem  # 

Fault 

1 

1 A 1 A6  Translator  Bypass  Module  bad 

2 

1A1 A23  5  MHz  Oscillator  signal  bad 

3 

1 A1A16  IF  Amplifier  Module  bad 

4 

Carrier  signal  from  1 A1 A8  Synthesizer  module  missing 

5 

Jumpers  are  not  installed  on  1A1TB4  ! 

The  hands-on  test  process  resulted  in  a  single  score  for  each  student  (i.e.,  the  student  problem 
score),  that  was  arrived  at  by  summing  the  scores  awarded  by  each  evaluator  on  each  problem, 
and  arriving  at  a  grand  mean  score  across  the  five  problems  and  two  evaluators.  A  weighting 
schema  was  used  within  each  problem  to  address  the  relative  importance  of  the  various 
troubleshooting  and  maintenance  activities  performed.  The  assessment  structure  is  addressed 
further  in  Appendix  A. 

Summary  Judgments  by  Evaluators.  The  evaluators  judged  student  summary  attributes,  in 
addition  to  the  test-items  under  the  six  dimensions  identified  above.  These  summary  judgments 
were  entered  onto  the  evaluation  form  after  the  student  completed  each  problem,  and  were 
analyzed  separately  from  the  detailed  task-item  scores.  The  summary  items  sought  the 
evaluators’  aggregate  judgments  of  the  student  performance  during  that  problem.  The  five  items 
were: 


Attributes:  Confidence 

Troubleshooting  process 
Needs  assistance 
Attitude 

Overall  Rating  (Proficiency) 

Problem  Time.  Each  evaluator  independently  recorded  the  start  and  stop  times  of  each  hands-on 
problem.  This  provided  a  measure  of  the  students’  troubleshooting  and  maintenance  efficiency. 

It  also  provided  a  means  of  checking  inter-rater  reliability,  since  their  start-stop  times  should 
have  been  similar. 

Although  the  hands-on  problem  assessment  approach  was  a  Level-2  evaluation,  according  to  the 
Kirkpatrick  model,  the  applied  hands-on  context  moved  it  beyond  the  typical  Level-2,  toward 
that  of  a  Level-3  evaluation. 
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2.1. 5.2  Training  Time 


The  time  spent  by  the  student  in  training  was  a  dependent  measure.  The  time  for  all  RS  group 
students  was  assumed  as  40  hours,  the  time  each  student  is  assigned  to  the  ANAVSC-3  resident 
course.  The  training  times  for  students  in  the  DS  and  RM  groups  were  those  recorded  by  the 
computer,  which  kept  a  log  of  the  times  the  ANAVSC-3  CBT  course  was  being  used. 

2.1.5.3  Knowledge  Tests  (Pretest  and  Posttest) 

The  Pretest  and  Posttest  were  developed  as  equivalent,  although  different,  versions  of  a  one-hour 
multiple-choice  test.  Each  test  consisted  of  two  sections:  1)  General  electronics;  and  2) 
ANAVSC-3.  The  general  electronics  section  assessed  the  student’s  level  of  general  knowledge 
about  electronics  theory,  operation,  troubleshooting  and  maintenance.  The  ANAVSC-3  section 
assessed  the  student’s  knowledge  about  similar  topics,  but  specifically  related  to  the  ANAVSC-3. 
The  number  of  questions  in  each  section  were  the  same  for  both  test  variants,  as  follows: 

General  Electronics:  19  questions 
ANAVSC-3:  27  questions 

The  Pretest  was  not  a  dependent  variable  of  the  experiment.  Its  purpose,  rather,  was  that  of  a 
controlling  factor,  providing  data  on  student  entry  proficiency.  The  Posttest,  on  the  other  hand, 
was  intended  as  a  measure  of  student  performance  resulting  from  the  training.  In  addition,  the 
difference  between  Posttest  and  Pretest  scores  for  the  ANAVSC-3  section  could  provide  data  for 
evaluating  training  gain.  The  Posttest  assessment  was  considered  a  Level-2  evaluation, 
according  to  the  Kirkpatrick  model. 

2.1.6  C-School  Critique 

Student  reaction  to  the  ANAVSC-3  course,  in  each  of  the  three  groups,  was  assessed  with  a 
student  questionnaire,  given  after  completion  of  all  training  and  testing  activities.  Three  versions 
of  the  questionnaires  were  developed,  one  for  each  group.  Each  of  the  three  questionnaires  was 
based  on  the  standard  USCG  C-School  Critique  (1996),  using  the  format  and  basic  set  of 
questions  from  that  document.  Additional  questions  were  added  for  each  group,  with  common 
questions  across  the  groups  when  feasible.  The  resulting  questionnaires,  therefore,  contained  a 
mix  of  data  including:  1)  data  that  were  relevant  only  to  the  particular  training  method,  2)  data 
that  were  comparative  across  the  two  CBT  training  methods,  3)  data  that  were  comparable  across 
the  three  training  methods,  and  4)  data  that  were  collected  to  maintain  commonality  with  the 
standard  USCG  C-School  Critique  (Questions  and  responses  relevant  to  only  this  last  category 
were  not  analyzed). 

The  modified  C-School  Critiques  were  considered  as  Level  1  evaluations,  under  the  Kirkpatrick 
model.  Each  of  the  critique  questionnaires  is  contained  in  Appendix  B. 

It  should  be  noted  that  some  of  the  survey  questions  were  included  to  maintain  close 
compatibility  with  the  standard  USCG  C-School  Critique,  and  some  questions  were  included  as  a 
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check  on  the  experiment  logistics  at  the  duty  stations.  Results  from  these  questions  were  not 
addressed  in  this  report. 

2.1.7  Other  Experiment  Materials 

Group-specific  materials  were  developed  to  support  the  experiment’s  training  and  testing 
activities.  These  materials  were  similar  across  the  groups,  for  similar  activities,  but  were  tailored 
somewhat  due  to  the  different  conditions  experienced  by  each  group  of  students.  Furthermore, 
the  material  requirements  differed  between  die  CBT  groups  (DS  and  RM)  and  the  RS  group,  due 
to  the  training  context.  The  materials  used  are  identified  in  the  subsections  that  follow  (See 
Appendix  C  for  additional  information  on  the  materials). 

2.1.7.1  Training  Materials 

The  three  student  groups  used  the  same  reference  publications  during  their  respective  ANAVSC- 
3  courses,  which  were: 

ANAVSC-3(v)7  Technical  Manuals  -  Two-volume  operation  and  maintenance  manual, 
developed  and  published  by  the  U.S.  Navy.  This  is  the  standard  reference  for  operating, 
troubleshooting  and  maintenance. 

ANAVSC-3(v)7  Student  Guide  -  Provides  troubleshooting  guidance  and  training-related 
information,  augmenting  the  technical  manuals.  This  document  was  developed  by  the 
USCG  specifically  to  support  the  resident  course. 

Student  Corrective  Maintenance  Log  Sheets  -  Blank  forms  used  by  students  to  write 
down  observations  and  other  information  during  the  troubleshooting  and  maintenance 
work.  All  students  were  instructed  to  use  these  forms  while  working  on  troubleshooting 
problems  in  the  course  (These  forms  were  also  used  by  the  students  during  the  hands-on 
testing  problems). 

The  DS  and  RM  groups  used  the  ANAVSC-3  CBT  medium,  in  addition  to  the  technical  manuals 
and  course  book.  The  computer  medium  contained  the  bulk  of  course  material,  which  was  stored 
on  the  laptop’s  hard  drive.  In  addition,  the  CBT  students  received  a  3-page  document,  Computer 
Turn-On/Off  Procedures,  to  guide  them  in  the  operation  of  the  laptop  computer. 

The  RS  group  used  the  standard  lab  equipment  and  other  media  in  the  ANAVSC-3  lab  at 
TRACEN  Petaluma  during  their  course. 

Additional  information  about  the  two  course  versions  is  provided  in  Section  2.1.2,  ANAVSC-3 
Course,  presented  earlier. 
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2.1. 7.2  Group-Specific  Materials 

The  group-specific  materials,  which  were  in  addition  to  the  training  materials  identified 
previously,  supported  conduct  of  the  experiment  for  the  three  groups  prior  to  the  TRACEN 
Petaluma  post-training  testing  activities.  They  are  identified  below  under  each  group. 

CBT  Dutv  Station  Group.  Materials  were  developed  for,  and  sent  to,  three  persons  at  each  duty 
station:  1)  Commanding  Officer;  2)  Facilitator  (the  person,  usually  the  student’s  immediate 
supervisor,  who  would  oversee  the  AN/WSC-3  CBT  course  conduct);  and  3)  Student.  The  set  of 
materials  consisted  of  the  following: 

Facilitator  Guideline  -  Handbook  guiding  facilitator  activities,  in  support  of  the  CBT 
and  information  collection  at  the  duty  station. 

Facilitator  Job  Aids  -  Several  aids,  such  as  a  check-off  sheet,  developed  to  assist  the 
facilitator  in  performing  the  requisite  tasks. 

Student  Guideline  -  One-page  document  providing  general  guidance  to  the  student. 

Duty  Station  Facilitator  Questionnaire  -  Survey  requesting  judgments  by  the  Facilitator 
about  the  CBT  course. 

Student  Background  Questionnaire  -  Survey  requesting  detailed  background 
information  about  each  student.  The  student  filled  out  this  questionnaire. 

Pretest  -  Paper-and-pencil  test,  equivalent  to  the  Posttest,  completed  by  the  student  prior 
to  start  of  the  training.  See  the  earlier  description. 

CBT  Remote  Group.  The  RM  group  used  the  same  materials  as  the  DS  group.  However, 
modifications  were  made  to  conform  to  administration  of  training  in  the  school  environment  at 
TRACEN  Petaluma. 

Resident  Group.  The  RS  group  materials  used  prior-to,  and  during,  the  training  consisted  of: 
Student  Background  Questionnaire  -  Identical  to  that  used  by  the  other  groups. 

Pretest  -  Identical  to  that  used  by  the  other  groups. 

Instructor  Guide  Supplement  -  Guidance  augmenting  the  standard  AN/WSC-3  Course ’s 
Instructor  Guide,  addressing  the  experiment.  This  guidance  supplement  addressed  the 
experiment. 
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2.1. 7.3  TRACEN  Petaluma  Evaluation  Materials 


The  post-training  testing  conducted  at  TRACEN  Petaluma  was  performed  the  same  for  each 
group,  and  used  the  same  materials,  as  follows: 

Faulted  AN/WSC-3  modules  -  Faulted  AN/WSC-3  modules  inserted  to  cause  the 
hands-on  problems. 

Evaluator  Guideline  -  Handbook  guiding  evaluator  activities  during  TRACEN  Petaluma 
testing. 

Scoring  Criteria  Guideline;  Scoring  sheet  -  Guide  and  scoring  sheet  used  by  evaluators 
to  judge  student  proficiency  during  hands-on  problems. 

AN/WSC-3  UHF  transceiver  and  lab  materials  -  AN/WSC-3  UHF  transceivers  and 
associated  equipment  and  tools,  used  during  the  hands-on  testing. 

2.1.8  Procedures 


The  sequence  of  experiment  activities,  which  was  the  same  for  all  groups,  is  presented  in 
Figure  1 .  The  first  four  activities  were  performed  at  the  respective  training  sites  for  each  group 
(e.g.,  at  the  individual  students’  duty  stations  for  the  DS  group).  The  final  three  activities  were 
performed  at  TRACEN  Petaluma  for  all  groups.  The  procedures  for  conduct  of  each  activity 
were  designed  to  be  similar,  in  keeping  with  the  context  of  each  group. 
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FIGURE  1.  MAJOR  EXPERIMENT  ACTIVITIES 
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2.1.8.1  Group  Sections  and  Subsections 


Each  group  (e.g.,  Duty  Station  Group)  was  divided  into  three  sections  of  no  more  than  6  students 
per  section.  As  can  be  seen  in  Table  3,  no  two  groups  or  sections  participated  in  the  hands-on 
testing  (evaluating  AN/WSC-3  troubleshooting  skills)  during  the  same  time  frame.  The  limit  on 
the  number  of  students  that  could  participate  in  any  one  section  was  based  on  size  of  the 
traditional  resident  class. 


TABLE  3.  TRAINING  AND  HANDS-ON  TESTING  SCHEDULE 


1 

TRAINING  PERIOD 

HANDS-ON  TESTING 

GROUP 

SECTION 

■ 

END 

BEGIN 

END 

DATE 

■ 

DATE 

DATE 

Resident 

1 

24  Feb  97 

28  Feb  97 

3  Mar  97 

4  Mar  97 

CBT  Duty  Station 

1 

24  Feb  97 

7  Mar  97 

1 1  Mar  97 

12  Mar  97 

CBT  Remote 

1 

17  Mar  97 

19  Mar  97 

20  Mar  97 

21  Mar  97 

Resident 

2 

28  Apr  97 

2  May  97 

5  May  97 

6  May  97 

CBT  Remote 

2 

19  May  97 

21  May  97 

22  May  97 

23  May  97 

Duty  Station 

2 

30  Jun  97 

11  Jul  97 

15  Jul  97 

16  Jul  97 

CBT  Remote 

3 

21  Jul  97 

23  Jul  97 

24  Jul  97 

25  Jul  97 

Resident 

3 

18  Aug  97 

22  Aug  97 

25  Aug  97 

26  Aug  97 

CBT  Duty  Station 

3 

25  Aug  97 

5  Sep  97 

9  Sep  97 

10  Sep  97 

To  facilitate  evaluator  observation  of  the  individual  students  during  the  hands-on  testing,  each 
section  was  further  divided  into  two  subsections  of  three  students  each.  The  five  hands-on  test 
problems  were  also  divided  up,  three  on  the  first  day  of  testing,  and  two  on  the  second  day  of 
testing.  Therefore,  one  subsection  would  take  the  first  three  problems  in  the  morning  (referred  to 
as  Subsection  A),  and  the  second  subsection  would  take  the  identical  three  problems  in  the 
afternoon  (referred  to  as  Subsection  B).  On  the  following  day  of  testing,  the  two  subsections 
would  swap  time  frames.  Subsection  B  would  take  the  final  two  problems  in  the  morning,  and 
Subsection  A  would  take  the  final  two  problems  in  the  afternoon.  Table  4  shows  the  order  for 
one  such  group  of  students. 
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TABLE  4.  HANDS-ON  TEST  PROBLEM  SCHEDULE 


SUB 

SECTION 

STUDENT 

1 

Test  Day  #1 

Problem  Start  Time 

1 

Test  Day  #2 

Problem  Start  Time 

0730 

0900 

1030 

1230 

1400 

1530 

1030 

1230 

1400 

1530 

A 

SI 

■ 

PI 

P2 

P3 

P5 

P4 

S2 

P2 

P3 

PI 

P4 

S3 

P3 

PI 

P2 

P5 

P4 

B 

S4 

P3 

P2 

PI 

P4 

P5 

S5 

P2 

PI 

P3 

P5 

P4 

S6 

_ 

PI 

P3 

P2 

MM 

P4 

P5 

S1-S6:  Student  identification  number  (6  students  per  section) 

P1-P5:  Problem  identification  number  (5  hands-on  problems) 


After  all  students  completed  a  problem,  they  would  move  to  another  ANAVSC-3  rack  and  start 
their  next  problem  (Table  4).  Students  were  prevented  from  talking  with  each  other  about  their 
previous  problems  to  maintain  testing  integrity.  To  control  for  potential  learning  effects  during 
the  testing,  the  order  in  which  students  performed  the  hands-on  test  problems  was  balanced 
within  and  across  subsections,  sections  and  groups. 

2.1.8.2  CBT  Duty  Station  Group  Procedures 

Each  unit  (duty  station)  for  which  a  unit  member  was  selected  to  participate  in  the  CBT  Duty 
Station  Group  was  contacted  initially  by  phone  one-month  before  the  training  period  was 
scheduled  to  begin.  The  phone  call  to  the  unit  was  made  by  either  a  representative  from  the 
USCG  Headquarters’  Office  of  Force  Management  (G-SRF),  or  a  representative  from  TRACEN 
Petaluma.  The  purpose  of  the  phone  call  was  to  inform  the  unit  of  the  selection,  to  establish 
points  of  contact  at  the  unit,  and  to  identify  a  facilitator  at  the  unit.  The  role  of  the  facilitator  was 
to  coordinate  experiment  support  activities  and  ANAVSC-3  training.  Following  the  phone 
contact,  a  letter  from  the  Commanding  Officer  of  the  R&DC  to  the  Commanding  Officer  of  the 
unit  was  sent.  The  purpose  of  the  letter  was  to  officially  request  the  unit’s  assistance  in 
performing  the  experiment,  and  to  inform  the  unit  as  to  what  would  be  required  of  them. 

One  week  before  the  training  period  was  scheduled  to  begin,  a  package  containing  a  set  of 
experiment  (e.g.,  knowledge  pretest)  and  training  (e.g.,  multimedia  laptop  computer  with 
courseware  loaded)  materials  would  arrive  at  the  participating  unit.  This  allowed  for  a  one-week 
preparation  period  during  which  the  facilitator  would  become  familiar  with  the  requirements  of 
the  experiment,  inventory  materials  in  the  package,  and  perform  a  test  of  the  laptop  computer.  If 
there  were  any  problems,  the  one- week  preparation  period  would  allow  for  resolution  without 
interfering  with  the  designated  training  period. 

A  two-week  CBT  training  period  was  specified  during  which  the  ANAVSC-3  course  would  be 
conducted.  On  the  first  day  of  the  training  period,  the  facilitator  administered  the  student 
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background  questionnaire  and  Pretest.  From  that  point  on,  the  student  was  to  perform  the  course 
at  the  student’s  own  pace.  The  student  was  requested  to  complete  the  course  by  the  last  day  of 
the  training  period.  The  facilitator  was  instructed  not  to  get  involved  in  the  student’s  training 
process. 

Immediately  following  completion  of  the  two-week  training  period,  the  student  was  sent  on 
Temporarily  Assigned  Duty  (TAD)  to  TRACEN  Petaluma  for  the  post-training  and  testing.  It 
should  be  noted  that  students  were  sent  TAD  even  if  they  did  not  complete  the  CBT  course,  and 
there  were  at  least  two  students  who  were  not  able  to  complete  all  of  the  CBT  course.  These  two 
students  experienced  an  unexpected,  and  extraordinarily  high,  workload  at  the  duty  station  during 
the  training  period.  As  with  all  the  students  in  this  group,  there  were  no  additional  opportunities 
for  these  students  to  complete  the  CBT  course  once  they  reported  to  the  TRACEN  Petaluma  lab 
for  the  post  training  and  testing  period. 

Six  DS  students,  at  different  duty  stations,  received  the  training  simultaneously,  and  reported 
together  at  TRACEN  Petaluma  for  the  testing.  The  procedures  for  the  DS  group  were: 

•  Contact  the  duty  station,  one-month  prior  to  training  activities. 

•  Reception  of  the  training  and  experiment  support  materials  one  week  before  start  of  the 
training  period. 

•  Pre-training  activities,  including: 

Student  Background  Questionnaire 
Pretest. 

•  Conduct  of  the  AN/WSC-3  CBT  course,  during  a  2  week  training  period.  Course 
completion. 

•  Student  travel  to  TRACEN  Petaluma  for  testing. 

•  Conduct  of  testing  at  TRACEN  Petaluma,  including: 

Hands-On  Training  and  Lab  Familiarization 
Posttest 

Hands-on  test  (5  problems) 

C-School  Critique 

2.1.8.3  CBT  Remote  Group  Procedures 

The  group  of  six  RM  students  reported  to  TRACEN  Petaluma  on  the  day  before  the  start  of 
training  and  experiment  activities,  similar  to  the  RS  students  (i.e.,  Sunday).  The  students 
gathered  Monday  morning  for  the  start  of  training,  in  a  classroom  remote  from  the  AN/WSC-3 
lab  where  hands-on  testing  would  be  conducted.  The  facilitator  provided  an  introduction,  and 
administered  the  student  background  questionnaire  and  Pretest  at  the  start  of  training.  The 
AN/WSC-3  CBT  course  then  commenced.  The  students  had  a  little  over  two  days  in  which  to 
complete  the  course,  before  the  start  of  testing  activities  late  Wednesday  morning.  The  students 
were  allowed  to  train  at  their  own  pace,  with  the  provision  that  they  had  to  finish  the  course  by 
late  Wednesday  morning. 

The  post-training  and  testing  process  began  late  Wednesday  morning,  and  concluded  Friday 
afternoon.  Hands-on  testing  was  conducted  on  Thursday  and  Friday.  This  testing  process  was 
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identical  to  that  of  the  DS  and  RS  groups,  with  the  exception  of  the  time  schedule,  which  differed 
for  each  group. 

The  procedures  for  the  RM  group  were: 

•  Pre-training  activities  at  TRACEN  Petaluma,  including: 

Student  Background  Questionnaire 
Pretest 

•  Conduct  of  the  AN/WSC-3  CBT  course,  during  a  two-plus  day  training  period.  Course 
completion. 

•  Conduct  of  testing  at  TRACEN  Petaluma,  including: 

Hands-On  Training  and  Lab  Familiarization 
Posttest 

Hands-on  test  (5  problems) 

C-School  Critique 

2.1. 8.4  Resident  Group  Procedures 

The  RS  students  reported  to  TRACEN  Petaluma  on  the  day  before  the  start  of  training  and 
experiment  activities,  similar  to  the  RM  students  (i.e.,  Sunday),  and  in  accordance  with  the 
normal  practice  for  the  traditional  resident  course.  The  students  began  training  Monday  morning, 
as  usual  for  the  AN/WSC-3  resident  course.  The  instructor  started  the  course  with  an 
introduction  to  the  experiment,  and  administered  the  student  background  questionnaire  and 
Pretest.  After  that  point,  and  until  the  conclusion  of  training  activities  Friday,  the  AN/WSC-3 
course  was  conducted  in  accordance  with  normal  practice,  with  no  deviations  for  experimental 
purposes. 

The  post-training  and  testing  process  began  Friday  afternoon,  and  concluded  during  Monday  and 
Tuesday  of  the  following  week.  The  hands-on  tests  were  conducted  Monday  and  Tuesday.  This 
process  was  identical  to  that  of  the  DS  and  RS  groups,  with  the  exception  of  the  time  schedule, 
which  differed  for  each  group. 

The  procedures  for  the  RS  group  were: 

•  Pre-training  activities  at  TRACEN  Petaluma,  including: 

Student  Background  Questionnaire 
Pretest. 

•  Conduct  of  the  resident  AN/WSC-3  course,  during  a  4- 1  /2  day  training  period.  Course 
completion. 

•  Conduct  of  testing  at  TRACEN  Petaluma,  including: 

Lab  Familiarization 
Posttest 

Hands-on  test  (5  problems) 

C-School  Critique 
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2.2  DUTY  STATION  IMPLEMENTATION  ANALYSIS  METHODOLOGY 


A  major  by-product  of  the  training  technology  experiment  was  a  group  of  USCG  duty  stations, 
and  staff,  that  had  first-hand  experience  with  nonresident  CBT  (i.e.,  through  participation  in  the 
DS  group).  Interviews  were  conducted  with  staff  at  a  subset  of  the  participating  duty  stations,  to 
generate  information  that  can  assist  the  USCG  in  effectively  implementing  nonresident  CBT 
training.  The  purpose  of  the  interviews  were  to: 

•  Identify  important  issues  and  potential  problems  associated  with  implementation  of 
nonresident  CBT  at  duty  stations. 

•  Identify  alternative  actions  for  dealing  with  the  issues  and  problems.  These  were  to  be 
generated  from  suggestions  during  the  interviews. 

2.2.1  Duty  Stations  Interviewed 

The  implementation  analysis  consisted  of  interviewing  relevant  staff  at  the  duty  stations.  The 
selection  of  duty  stations  was  constrained  to  assure  interviews  with  East  Coast,  Gulf  Coast  and 
West  Coast  units.  The  interviews  were  conducted  shortly  after  completion  of  all  experimental 
training  and  testing.  By  so  doing,  the  interview  discussions  did  not  affect  the  training  or  testing 
activities. 

Interviews  were  conducted  over  a  period  of  about  2-1/2  months,  with  the  following  duty  stations: 

East  Coast:  Engineering  Support  Unit,  Portsmouth,  VA 

Medium  Endurance  Cutter  CGC  DEPENDABLE,  Portsmouth,  VA 
Research  &  Development  Center,  Groton,  CT 

Gulf  Coast:  Engineering  Support  Detachment,  Corpus  Christi,  TX 

West  Coast:  Engineering  Support  Unit,  Honolulu,  HA 

Ice  Breaker  CGC  POLAR  STAR,  Seattle,  WA 
Sea-Going  Buoy  Tender  CGC  COWSLIP,  Astoria,  OR 

These  duty  stations  comprised  about  half  of  the  units  at  which  the  AN/WSC-3  CBT  course  was 
conducted,  and  about  60%  of  the  DS  group  students. 

2.2.2  Duty  Station  Staff  Interviewed 


The  primary  staff  to  be  interviewed  at  each  duty  station  were  (Note,  not  all  units  had  an 
Education  Officer  and  Training  Officer): 

Commanding  Officer  or  Executive  Officer 

Student’s  immediate  supervisor  (who  often  was  the  facilitator) 

Education  Officer  and/or  Training  Officer 
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Other  appropriate  persons  were  also  interviewed,  such  as  the  students.  A  total  of  27  interviews 
were  conducted.  Most  interviews  were  with  one  person  at  a  time;  some  interviews  were  with 
multiple  persons. 


2.2.3  Interview  Form.  Content  and  Process 

A  structured  interview  form  was  developed  to  identify  the  major  interview  dimensions,  and  list 
the  questions.  It  contained  53  questions,  with  sub-questions,  grouped  under  the  following 
dimensions: 

AN/WSC-3  CBT:  Overall 

Duty  Station:  Training  Space  &  Environment 
Time  Available  to  Train 
Training  Disruptions  &  Interference 
Training  Management 
Student 

Training  Process 
Organization  Issues 


The  interview  form  was  intended  to  provide  guidance  only,  as  opposed  to  a  constrained  interview 
procedure.  The  goal  was  to  conduct  the  interviews  in  a  conversational  format,  allowing  for  the 
interviewee  to  raise  important  issues  and  discuss  ideas.  Hence,  the  order  of  the  questions 
differed  somewhat  during  each  interview  based  on  the  responses  from  the  interviewee. 

The  interviews  were  conducted  with  two  interviewers  present:  one  representative  from  Paradigm 
Associates,  and  one  representative  from  R&DC.  Usually,  the  interviews  were  with  one  duty 
station  person  at  a  time;  some  interviews  were  with  multiple  persons  in  a  small  group  setting. 
The  interviews  ranged  in  time  from  about  1  to  2  hours,  generally  requiring  about  1-1/2  hours  to 
complete. 

3  TRAINING  EFFECTIVENESS  RESULTS 

The  results  obtained  from  each  of  the  student  performance  measures  and  the  other  evaluation 
instruments  are  presented  in  this  section  of  the  report. 

3.1  HANDS-ON  TROUBLESHOOTING  PERFORMANCE 

Student  performance  during  the  hands-on  problems  was  considered  the  single  most  important 
indicator  of  training  effectiveness,  since  it  required  applied  student  performance  in  an 
environment  closely  replicating  that  of  an  operational  unit.  Each  of  the  46  students  were  scored 
on  all  five  hands-on  problems  by  the  two  evaluators,  in  accordance  with  the  specified  scoring 
process,  yielding  a  total  of  more  than  8000  assessment  data  points  across  the  students. 

The  analysis  of  the  hands-on  problem  data  was  performed  in  three  parts,  with  a  fourth  part 
addressing  the  evaluators’  reliability. 
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3.1.1  Student  Problem  Scores 


The  resulting  mean  problem  scores  achieved  by  each  group  are  presented  in  Table  5.  The  mean 
problem  score  is  the  average  of  student  scores  for  each  problem,  in  each  group  (An  explanation 
of  the  scoring  process  is  in  Section  2. 1 .5,  Performance  Measurement  Tools;  and  in  Appendix  A, 
Hands  on  Problems,  Description  and  Scoring).  Student  scores  on  individual  problems  ranged 
from  about  57%  to  100%.  As  can  be  seen,  the  mean  problem  scores  for  each  group  were  near 
90%.  The  RM  group  achieved  the  highest  mean  score,  closely  followed  by  the  DS  group,  and  the 
RS  group.  A  one-way  analysis  of  variance  (ANOVA)  design  was  performed  to  evaluate  group 
differences,  with  the  following  results: 


•  A  significant  main  effect  was  found  for  the  student  group  variable  (training-delivery 
method),  F(2, 43)  =  3.818,  p  <  0.03. 

•  A  significant  difference  was  found  between  the  RM  and  RS  groups  (p  <  0.05),  using  the 
Tukey  H  test  to  evaluate  specific  group  differences.  The  RM  group  performed  better  than 
the  RS  group. 

•  No  significant  differences  were  found  between  other  group  comparisons  (i.e.,  DS  versus 
RS,  DS  versus  RM,  and  CBT  (DS  +  RM)  versus  RS). 


TABLE  5. 


MEAN 


HANDS-ON  PROBLEM  SCORES  FOR  EACH  STUDENT  GROUP 


Hands-on 

Problem  # 

CBT 

Duty  Station 
(DS) 

Student  Group 
CBT 
Remote 
(RM) 

Resident 

(RS) 

1 

91.92 

92.35 

87.89 

2 

92.36 

93.80 

90.42 

3 

91.83 

87.68 

92.01 

4 

88.16 

91.46 

83.54 

5 

94.78 

94.74 

85.48 

Mean 

91.81 

92.00 

87.87 

These  results  indicate  that  the  AN/WSC-3  CBT  course,  both  at  the  duty  station  and  at  the 
training  center,  was  as  effective  as  the  instructor-led  AN/WSC-3  resident  course. 

3.1.2  Summary  Evaluator  Ratings 

The  mean  evaluator  ratings  of  the  summary  attributes  for  students  in  each  group  are  presented  in 
Table  6.  These  represent  the  bottom-line  evaluator  judgments  of  each  student,  following 
completion  of  each  hands-on  problem.  The  tabulated  ratings  are  the  group  means,  calculated 
across  students  and  problems  for  each  group.  Inspection  of  this  table  shows  that  the  ratings  were 
similar  across  the  three  groups  for  each  attribute.  The  order  of  the  ratings  were  consistent  for 
each  attribute,  with  the  RM  group  achieving  the  highest  mean,  closely  followed  by  the  DS  group, 
and  the  RS  group.  No  significant  difference  was  found  between  groups  for  any  of  the  attributes, 
based  on  a  one-way  ANOVA  performed  on  the  data  of  each  attribute,  as  follows: 
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•  Confidence,  F(2,43)  =  0. 146,  p  <  0.864 

•  Troubleshooting  Process  F(2,43)  =  0.667,  p  <  0.87 

•  Assistance  Need  F(2,43)  =  0.8 1 5,  p  <  0.449 

•  Attitude  F(2,43)  =  0. 1 1 9,  p  <  0.888 

•  Overall  Rating  F(2,43)  =  0.837,  p  <  0.44 


TABLE  6.  MEAN  HANDS-ON  PROBLEM  SUMMARY  RATINGS  FOR  EACH 


Summary 

Attribute 

CBT 

Duty  Station 
(DS) 

Student  Group 
CBT 
Remote 
(RM) 

Resident 

(RS) 

Confidence 

3.04 

3.09 

2.98 

Troubleshooting 

3.02 

3.07 

2.89 

Assistance  Need 

2.94 

3.06 

2.85 

Attitude 

3.54 

3.60 

3.53 

Overall  Rating 

2.66 

2.81 

2.55 

These  results  indicate  that  the  students  receiving  the  AN/WSC-3  CBT  course,  both  at  the  duty 
station  and  at  the  training  center,  exhibited  attributes  equivalent  to  graduates  of  the  instructor-led 
AN/WSC-3  resident  course. 

3.1.3  Problem  Solving  Time 

The  students’  problem-solving  times  are  presented  in  Table  7.  The  times  are  the  group  means, 
calculated  in  minutes,  across  students  and  problems.  The  data  show  that  the  shortest  mean  time 
was  achieved  by  the  RS  group,  followed  by  the  RM  group,  and  the  DS  group.  No  significant 
differences  in  problem-solving  time  were  found  between  groups  (Based  on  a  one-way  ANOVA, 
F(2,43)  =  2.57,  p  <  0.088).  However,  the  closeness  to  significance  suggests  a  trend  may  be 
present  in  the  data,  meaning  that  a  significant  difference  may  exist;  additional  research  is 
necessary  to  establish  if  a  difference  does  exist.  Nevertheless,  the  closeness  of  the  problem 
solving  times  suggests  that,  if  a  difference  were  later  found,  it  would  be  relatively  small. 


TABLE  7.  MEAN  TIME  TO  SOLVE  THE  HANDS-ON  PROBLEMS,  FOR  EACH 
GROUP  , 


CBT 

Duty  Station 
(DS) 

Student  Group 
CBT 
Remote 
(RM) 

Resident 

(RS) 

Problem 
Solving 
Time  (min.) 

58.89 

56.48 

51.15 
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3.1.4  Inter-Evaluator  Reliability 


The  reliability  of  the  student  evaluations,  performed  by  the  two  evaluators,  was  estimated  by:  1) 
correlating  the  student  problem  scores  between  the  two  evaluators;  and  2)  correlating  the 
problem  solving  times  between  the  two  evaluators.  The  correlation  of  problem  scores  was 
expected  to  yield  a  reasonable  estimate  of  the  inter-evaluator  reliability,  as  regards  the  hands-on 
problem  scores.  The  correlation  of  the  problem  solving  times,  on  the  other  hand,  was  viewed  as 
a  check  on  the  evaluators’  reliability,  but  not  a  reasonable  estimate  of  the  student  problem  score 
reliability.  This  later  correlation  should  be  high  if  the  evaluators  performed  reliably,  since  the 
problem  solving  times  should  be  very  similar  between  evaluators.  The  results  of  these  analyses 
were; 

•  The  inter-evaluator  score  correlation  for  the  hands-on  problems  was  found  to  be  moderate 
to  high  (r  =  0.71),  and  highly  significant  (p  <  0.001). 

•  The  inter-evaluator  problem  solving  time  correlation  was  very  high  (r  =  0.94),  and  highly 
significant  (p  <  0.001). 

These  results  indicate  that  the  hands-on  problems’  evaluation  process  conducted  at  TRACEN 
Petaluma  was  satisfactorily  reliable. 

3.2  STUDENT  KNOWLEDGE 


Student  knowledge,  as  measured  by  the  Pretest  and  Posttest,  is  reported  in  this  subsection.  All 
students  in  each  group  completed  both  the  Pretest  and  the  Posttest.  However,  certain 
discrepancies  occurred  during  administration  of  the  tests,  which  constrained  the  analysis  and 
interpretation  of  results. 

•  The  DS  group  scores  on  the  AN/WSC-3  part  of  the  Pretest  were  not  in  conformance  with 
those  of  students  in  the  other  two  groups.  Hence,  the  usefulness  of  Pretest  data  was 
limited.  This  was  most  probably  due  to  a  combination  of  factors,  including  the  duty 
station  environment  (e.g.,  lack  of  tight  experimental  control). 

•  The  posttesting  of  the  final  section  of  DS  students,  which  was  also  the  final  section  of 
students  tested,  had  to  be  accomplished  two  to  four  weeks  after  the  students  completed 
the  course  (i.e.,  compared  with  several  hours  to  2  days  later  for  the  other  students  in  all 
groups).  This  posttesting  delay  introduced  potential  contamination  into  the  data  (e.g., 
retention  effect). 

For  these  reasons,  only  partial  analyses  of  the  knowledge  tests  were  performed.  Table  8  presents 
the  mean  raw  scores  achieved  on  the  Pretest  and  Posttest,  including  the  General  and  AN/WSC-3 
part-scores.  Note,  the  raw  scores  are  the  number  of  questions  answered  correctly.  Also,  note  that 
the  number  of  questions  differed  on  each  part  (i.e.,  19  questions  on  the  General  part,  and  27 
questions  on  the  AN/WSC-3  part).  The  tests  were  constructed  to  be  difficult,  thus  providing  a 
high  potential  for  achieving  meaningful  differences  between  the  Pretest  and  Posttest,  if  training 
was  effective.  The  “Posttest  -  Pretest”  column  represents  the  difference  between  Posttest  and 
Pretest  scores. 
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TABLE  8.  PRETEST  AND  POSTTEST  SCORES 


Test  Part 

Training  Group 

Pretest 

Posttest 

Posttest  - 
Pretest 

General 

(Out  of  19  questions) 

CBT  Duty  Station 

10.82 

11.41 

0.59* 

CBT  Remote 

9.44 

10.94 

1.50 

Resident 

10.15 

11.62 

1.47 

AN/WSC-3 

(Out  of  27  questions) 

CBT  Duty  Station 

9.71 

17.00 

7.29* 

CBT  Remote 

6.81 

16.13 

9.32 

Resident 

5.54 

18.46 

12.92 

TOTAL  SCORE 
(General  &  AN/WSC-3) 

CBT  Duty  Station 

20.53 

28.41 

7.88* 

CBT  Remote 

16.25 

27.06 

10.82 

Resident 

15.69 

30.08 

14.39 

♦Note:  These  values  may  reflect  discrepancies  that  occurred  d 

uring  administration 

of  the  pre-  and  post-tests  of  the  Duty  Station  Group.  (See  section  3.2) 

3.2.1  Student  Entry  Characteristics 

Inspection  of  Table  8  shows  that  the  general  student  entry  characteristics,  as  measured  by  the 
general  part  of  the  Pretest,  were  stronger  than  those  related  to  the  AN/WSC-3.  The  mean  score 
on  the  Pretest-general  part,  across  all  three  groups,  was  about  53%  (The  group  means  were 
between  50%  and  57%).  In  contrast,  the  mean  score  on  the  Pretest- AN/WSC-3  part,  across 
groups,  was  about  28%.  These  relative  student  characteristics  were,  of  course,  expected.  They 
indicate  students  had  little  knowledge  on  how  to  operate  and  maintain  a  AN/WSC-3(v)7  UHF 
transceiver  prior  to  the  course. 

3.2.2  Training  Gain 

The  difference  in  student  scores  between  the  Pretest  and  Posttest,  particularly  on  the  AN/WSC-3 
part,  provided  a  measure  of  training  gain  during  the  course.  The  resident  course,  based  on 
discussions  with  USCG  personnel,  was  assumed  to  be  effective,  and  thus  was  the  yardstick  for 
comparison  with  the  two  CBT  courses.  Repeated  measures  ANOVA’s  were  performed  on  the 
Pretest  and  Posttest  scores  for  each  group,  to  evaluate  training  gain.  The  data  problems 
encountered  with  these  tests,  as  noted  above,  acted  to  reduce  the  observed  training  gain  for  the 
DS  group;  hence,  this  analysis  was  considered  valid,  and  conservative. 

3.2.2.1  AN/WSC-3  Part  of  Test 

The  AN/WSC-3  part  of  each  test  was  of  most  interest,  since  it  was  the  focus  of  the  course. 
Significant  training  gains  were  found  in  the  AN/WSC-3  part  or  each  group,  based  on 
performance  improvement  from  the  Pretest  to  the  Posttest  (RS  group,  F(l,12)  =  208.5,  p  <  0.001; 
DS  group,  F(l,16)  =  26.05,  p  <  0.001;  RM  group,  F(l,15)  =  84.12,  p  <  0.001).  These  results 
show  that  the  students  in  each  of  the  three  training  method  groups  performed  significantly  better 
after  completing  the  respective  courses.  The  magnitude  of  the  changes,  as  shown  in  Table  8, 
indicates  substantial  increase  in  student  knowledge  as  a  result  of  each  course. 
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3.2.2.2  General  Part  of  Test 


The  general  part  of  each  test  was  expected  to  show  only  a  small  gain  between  the  Pretest  and 
Posttest,  since  the  AN/WSC-3  course  only  indirectly  addressed  general  electronics  topics.  The 
mean  scores  of  the  general  part  of  the  Posttest  show  a  small  increase  over  those  of  the  Pretest,  for 
each  group.  No  significant  differences  were  found  between  the  Pretest  and  Posttest  in  the  general 
part,  for  any  of  the  groups  (RS  group,  F(l,  12)  =  2.892,  p  <  0.1 15;  DS  group,  F(l,  16)  =  0.402,  p 
<  0.535;  RM  group,  F(l,  15)  =  3.553,  p  <  0.079). 

3.2.3  Post-Training  Knowledge 

The  Posttest  scores  were  analyzed  to  determine  if  significant  group  differences  existed  in  student 
knowledge  after  completing  the  course.  This  analysis  was  questionable,  because  of  the  possible 
data  contamination  noted  earlier.  No  significant  differences  were  found  between  the  three 
training  methods  (Posttest-general  part,  F[2,  43]  =  0.267,  p  <  0.767;  Posttest- AN/WSC-3  part, 
F[2,  43]  =  2.288,  p  <  0.1 14). 

3.3  STUDENT  REACTION  TO  THE  AN/WSC-3  CBT  COURSE 


This  subsection  addresses  the  students’  reactions  to  the  AN/WSC-3  CBT  course,  and  to  training 
at  duty  stations.  The  critique  results  are  both  presented  and  discussed  in  this  section. 

The  investigation  of  student  reactions  to  the  course  centered  around  those  of  the  DS  group,  with 
the  focus  on  duty  station  training.  The  RM  group  reactions  were  important  with  regard  to  their 
similarity  or  difference  in  comparison  with  those  of  the  DS  group,  particularly  as  related  to  the 
training  environment.  The  reactions  of  the  RS  group  were  important  in  comparison  with  the 
combined  DS  and  RM  group  reactions,  providing  data  related  to  the  comparison  of  traditional 
training  methods  with  CBT. 

The  student  response  frequencies  for  all  Likert-type  questions  in  the  C-School  Critiques,  for  each 
of  the  three  student  groups,  are  listed  in  Appendix  E.  Some  questions,  which  were  included  in 
the  critique  to  be  compatible  with  the  standard  USCG  C-School  Critique,  were  not  of  specific 
interest  to  this  study  and  are  not  discussed. 

Note,  the  findings  reported  in  this  section  often  compliment  those  obtained  from  the  duty  station 
interviews,  which  are  addressed  Section  5,  Duty  Station  Implementation  Analysis  Findings. 

3.3.1  Reaction  to  Duty  Station  Training 

Student  responses  to  the  44  Likert-type  questions  and  the  6  open-ended  questions  of  the  DS 
Group’s  critique  were  analyzed  in  sets  of  related  questions. 
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3.3.1. 1  Effectiveness  of  ANAVSC-3  CBT  Course 


The  DS  group  responses  to  the  4  Likert-type  questions  that  addressed  the  training  effectiveness 
of  the  ANAVSC-3  CBT  course  are  presented  in  Table  9.  The  number  of  students  that  selected 
each  question  response  category  (i.e.,  frequency),  is  listed  for  each  question.  The  statement 
associated  with  each  question  number  in  the  table  paraphrases  the  actual  question  (The  course 
critiques  are  in  Appendix  B,  containing  the  precise  statement  of  each  question). 


TABLE  9.  DS  GROUP  JUDGMENTS  ABOUT  EFFECTIVENESS  OF  ANAVSC-3 
TRAINING _ _ _ 


Questions  Regarding  Effectiveness  of  Training 

Question  Response  Categories 
(Frequency  of  Student  Responses) 

N/A 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

1 .  Confident  in  ability  to  maintain  ANAVSC-3 

0 

0 

0 

12 

5 

26.  Training  quality  was  satisfactory 

0 

0 

3 

11 

3 

27.  Course  was  effective  overall  * 

0 

0 

0 

13 

3 

28.  CBT  course  as  effective  as  traditional  course 

0 

1 

9 

5 

2 

32.  Computer  satisfactory  for  practice 

0 

1 

3 

11 

2 

Note:  *  denotes  one  or  two  students  failed  to  answer  question 


Inspection  of  Table  9  shows  that  the  large  majority  of  students  agreed  with  the  statements. 

Nearly  80%  of  the  student  responses  (i.e.,  67  of  the  84  responses)  were  in  the  “Agree”  or 
“Strongly  Agree”  categories,  indicating  that  most  students  judged  the  CBT  course  to  be  effective. 
All  students  agreed  that  the  CBT  course  was  effective  overall,  and  that  they  were  confident  in 
their  ability  to  maintain  the  ANAVSC-3(v)7  UHF  transceiver  (questions  #27  and  #1).  Most 
students  judged  the  training  to  be  satisfactory,  although  several  (about  18%)  disagreed  (question 
#26).  Similarly,  most  students  judged  the  computer  as  satisfactory  for  practice,  although  about  a 
quarter  of  the  students  disagreed,  with  one  student  strongly  disagreeing  (question  #32).  In 
contrast  to  these  findings,  less  than  half  the  students  (about  41%)  judged  the  CBT  course  to  be  as 
effective  as  a  traditional  course  (question  #28).  This  DS  group  finding  differs  substantially  with 
reactions  of  RM  group  students,  all  of  whom  agreed  or  strongly  agreed  that  the  CBT  course  was 
as  effective  as  a  traditional  course.  It  is  suggested,  therefore,  that  the  disagreeing  DS  student 
responses  to  question  #28  reflect  specific  difficulties  that  those  students  experienced  during 
implementation  of  the  CBT  at  their  respective  duty  stations.  This  issue  is  addressed  further  in  a 
following  subsection  (Section  3.3.2,  Comparison  of  Duty  Station  Student  Reactions  with  Other 
Groups). 

The  results  presented  in  Table  9  generally  agree  with  those  obtained  from  the  hands-on  tests,  and 
the  Pretest  and  Posttest.  Most  students  judged  the  ANAVSC-3  CBT  course  to  be  satisfactory  and 
effective. 
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3 .3.1.2  AN/WSC-3  CBT  Course  Content 


Student  responses  to  the  14  questions  addressing  the  content  of  the  AN/WSC-3  CBT  course  are 
presented  in  Table  10.  With  the  exception  of  question  #3,  the  large  majority  of  responses  agreed 
or  strongly  agreed  with  the  statements  (i.e.,  188  of  213  responses,  or  88%)  (question  #3 
responses,  and  the  4  responses  in  the  “N/A”  category  were  not  included  in  this  calculation).  This 
indicates  very  positive  student  reactions  to  most  of  the  course  content.  In  contrast,  most  students 
judged  that  additional  practice  was  needed  during  the  course,  as  indicated  by  the  disagreeing 
responses  to  question  #3  (i.e.,  11  of  16  responses,  or  69%).  This  need  for  additional  practice  was 
echoed  in  the  open-ended  responses,  as  addressed  below,  and  also  by  the  RM  group.  With  the 
exception  of  practice,  therefore,  the  AN/WSC-3  CBT  course  content  was  judged  favorably  by  the 
students.  This  finding  coincides  with  the  students’  judgment  of  the  course  as  effective. 


TABLE  10.  DS  GROUP  JUDGMENTS  ABOUT  DESIGN  AND  IMPLEMENTATION  OF 
THE  AN/WSC-3  CBT  COURSE 


Questions  Regarding  Design  and  Implementation  of  Course 

N/A 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

3.  Practice  was  sufficient 

0 

1 

10 

6 

0 

4.  Proper  troubleshooting  techniques  were  taught 

0 

0 

2 

13 

2 

5.  Troubleshooting  problems  were  realistic 

0 

0 

0 

14 

3 

6.  Course  was  presented  in  a  logical  sequence 

0 

0 

0 

14 

3 

7.  Course  length  was  ideal  * 

0 

0 

4 

12 

0 

8.  Proper  tools  &  equipment  available  on  computer 

1 

1 

1 

12 

2 

9.  Spare  parts  available  in  computer  medium  * 

1 

0 

1 

11 

2 

13.  HELP  information  enhanced  learning 

0 

0 

0 

12 

5 

14.  Course  work  related  to  course  objectives 

0 

0 

0 

15 

2 

23.  Training  time  was  never  wasted 

1 

1 

3 

9 

3 

29.  Instruction  detail  was  sufficient  * 

0 

0 

4 

11 

1 

30.  Course  was  challenging 

0 

0 

4 

13 

0 

3 1 .  Amount  of  general  theory  was  sufficient 

0 

1 

3 

11 

2 

35.  Computer  &  software  worked  satisfactorily 

1 

0 

0 

7 

9 

Note:  *  denotes  one  or  two  students  failed  to  answer  question 


The  student  responses  to  the  open-ended  questions  provided  information  amplifying  on  the 
results  in  Table  10.  The  following  were  cited  as  the  most  beneficial  parts  of  the  CBT  course 
(question  #45): 

•  Inherent  characteristics  of  duty  station  CBT,  such  as: 

Self-paced  nature  of  CBT 

Graphical  presentation 

Learning  at  the  student’s  duty  station 

AN/WSC-3  information  access  at  any  time 

Able  to  review  at  any  time 
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•  Time  savings  that  accrue  from  CBT 

•  Aspects  of  the  AN/WSC-3  course,  such  as: 

Troubleshooting  methods 
BITE  test  explanation  and  faults 
Knowing  which  boards  to  check 

Despite  the  very  positive  student  response  to  the  CBT  course,  several  areas  of  improvement  were 
identified.  Insufficient  practice  problems  were  cited,  by  several  students,  as  the  least  beneficial 
part  of  the  course  (question  #46).  Furthermore,  nearly  half  the  students  cited  the  need  for  more 
practice  problems  in  response  to  question  #47,  which  asked  about  additional  information  and 
materials  for  improving  the  course.  The  need  for  more  practice  problems  was  again  cited  under 
question  #50,  which  sought  suggestions  for  improving  CBT  at  duty  stations.  In  general,  across 
all  questions,  the  need  for  additional  practice  problems  was  the  most-often  cited  improvement 
wanted  for  the  AN/WSC-3  CBT  course.  These  results,  together  with  the  predominant 
disagreeing  response  to  question  #3,  clearly  identify  the  need  for  more  practice  problems  in  the 
CBT  course. 

Several  other  improvement  needs,  cited  by  one  or  more  students  in  response  to  questions  #46,  47 
and  50,  are  listed  below,  in  order  of  how-often  cited: 

•  Provide  additional  theory,  such  as  how  modules  work,  operation  of  individual  circuits, 
power  distribution,  troubleshooting  schematics,  and  signal  tracing. 

•  Provide  hands-on  training  at  the  duty  station,  rather  than  at  the  training  center  following 
completion  of  the  CBT  part. 

•  Improve  specific  CBT  characteristics,  such  as  a  more  convenient  means  of  backtracking 
during  training. 

•  Remove  non-essential  parts  of  the  CBT  course,  such  as  the  salty-chief  cartoon  character. 

•  Provide  a  means  for  the  student  to  interact  with  an  instructor. 

An  important  aspect  of  the  CBT  course  content  was  its  ease  of  use  (i.e.,  user  interface).  Student 
judgments  on  the  three  questions  addressing  the  user  interface  are  presented  in  Table  1 1.  As  can 
be  seen,  the  students  judged  the  user  interface  very  favorably,  with  only  a  small  proportion  of 
students  disagreeing  (i.e.,  47  of  49  responses  agreed  or  strongly  agreed,  representing  96%)  (the  2 
“N/A”  responses  were  not  included).  This  result  suggests  that  the  user  interface  likely 
contributed  to  the  effectiveness  of,  and  complimented,  the  course  content. 


TABLE  11.  DS  GROUP  JUDGMENTS  RELATED  TO  THE  USER  INTERFACE 


Question  Response  Categories 

Questions  Regarding  User  Interface 

(Frequency  of  Student  Responses) 

|  N/A 

Strongly 

Disagree 

Agree 

Strongly 

Disagree 

Agree 

34.  CBT  materials  were  user  friendly 

1  1 

0 

1 

6 

9 

36.  I  had  no  difficulty  learning  to  use  computer 

0 

0 

0 

5 

12 

41.  I  had  no  difficulty  conducting  the  CBT 

1 

0 

1 

11 

4 

29 


The  above  findings  provide  evidence  of  a  satisfactory  and  effective  AN/WSC-3  CBT  course 
having  been  conducted  at  duty  stations.  They  also  indicate  that  some  students  had  difficulty 
training,  identify  areas  of  potential  improvement  to  this  CBT  course,  and  identify  areas  that 
should  be  given  more  attention  in  future  courses  (e.g.,  practice  problems,  and  hands-on  training 
at  duty  stations). 


3.3.1.3  Duty  Station  Environment 

The  duty  station  environment  was  the  context  in  which  the  AN/WSC-3  CBT  course  was 
conducted  for  DS  group  students,  and  in  which  future  nonresident  training  (e.g.,  CBT,  IVT)  may 
be  conducted.  Consequently,  the  student  reactions  to  their  training  environment  are  of  particular 
importance.  Student  responses  to  the  eight  questions  addressing  the  duty  station  training 
environment  are  presented  in  Table  12.  In  aggregate,  the  majority  of  students  agreed  with  the 
statements,  indicating  a  generally  positive  student  reaction  to  the  training  environment  at  duty 
stations  (i.e.,  92  of  132  responses  agreed  or  strongly  agreed,  representing  70%  of  the  responses) 
(the  4  “N/A”  responses  were  not  included).  Some  students,  however,  disagreed  or  strongly 
disagreed  with  each  statement,  suggesting  considerable  differences  in  the  training  environments 
across  the  DS  group.  More  importantly,  the  results  suggest  that  30%  of  the  students,  on  average, 
judged  aspects  of  their  training  environment  as  not  conducive  to  training.  This  finding  has 
important  ramifications  for  effective  implementation  of  nonresident  training,  highlighting  the 
need  to  carefully  design,  and  control,  the  duty  station  training  environment. 


TABLE  12.  DS  GROUP  JUDGMENTS  RELATED  TO  THE  TRAINING 
ENVIRONMENT  _ 


Questions  Regarding  Training  Environment 

Question  Response  Categories 
(Frequency  of  Student  Responses) 

N/A 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

20.  Training  environment  was  functional 

1 

2 

2 

9 

3 

21 .  Good  visibility  in  training  space 

0 

3 

10 

4 

22.  No  distractions  during  training 

■■ 

2 

5 

7 

3 

37.  Training  environment  was  satisfactory 

0 

0 

5 

8 

4 

38.  Available  time  for  course  was  adequate 

0 

1 

5 

8 

3 

40.  Command  supported  AN/AN/WSC-3  CBT 

2 

0 

1 

8 

6 

42.  CBT  at  duty  station  was  convenient 

1 

6 

6 

4 

43.  Little  interference  during  training 

1 

6 

5 

4 

The  most  positive  responses  in  this  set  of  questions  were  given  to  question  #40,  which  addressed 
command  support  for  the  experimental  CBT  course.  All  but  one  student  agreed  that  their 
command  supported  the  training,  an  important  finding.  Nevertheless,  although  the  command 
support  was  widespread,  the  distractions,  interference  and  available  training  time  apparently  were 
not  always  favorable  to  the  duty  station  training. 

Information  in  the  open-ended  questions  amplified  on  these  findings.  Interruptions  were  cited, 
by  several  students,  as  the  least  beneficial  part  of  the  CBT  course  (question  #46).  More 
importantly,  the  predominant  response  to  question  #50  (i.e.,  suggestions  for  improving  CBT  at 
duty  stations)  cited  factors  directly  related  to  management  of  the  training/work  environment  as 
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the  key  to  nonresident  training  success.  These  included  elimination  of  interruptions;  integration 
of  training,  and  work  prioritizing  and  scheduling;  and  increasing  command  emphasis  on  training. 
Two-thirds  of  the  students  cited  the  need  to  develop  a  strong  command  commitment  to  duty 
station  training,  and  thus  reduce  interference  to  students  during  training.  These  findings,  together 
with  those  cited  earlier,  suggest  that  widespread  command  support  by  itself  is  not  enough  to 
assure  an  adequate  training  environment.  The  duty  station  training  process  must  be  purposely 
configured  and  actively  controlled.  This  is  a  fundamental  issue  that  must  be  resolved  if 
nonresident  training  is  to  succeed  in  a  duty  station  training  environment. 

3.3.1. 4  Facilitator  Functioning 

The  facilitator’s  functions  at  the  duty  stations  were  primarily  in  support  of  the  experiment,  rather 
than  the  CBT  course.  However,  large-scale  implementation  of  nonresident  training  (e.g.,  CBT, 
IVT)  at  duty  stations  will  require  some  level  of  oversight  by  a  training  coordinator,  such  as  the 
facilitator  used  in  this  experiment.  Student  judgments  about  the  facilitator,  therefore,  provided 
insight  into  the  potential  functioning  of  a  duty  station  training  coordinator. 

Student  responses  to  the  4  questions  pertinent  to  facilitator  functioning  are  presented  in  Table  13. 
Although  the  median  responses  to  all  questions  were  in  the  “Agree”  category,  student  reactions 
differed  considerably  on  each  question.  Between  53%  and  71%  of  the  students  agreed  with  the 
statements,  representing  an  aggregate  of  62%  agreeing  or  strongly  agreeing  (i.e.,  40  of  65 
responses  agreed  or  strongly  agreed)  (the  3  “N/A”  responses  were  not  included).  This  result 
suggests  differences  of  opinion  among  the  students  regarding  facilitator  functioning.  This 
finding  is  complimentary  to  the  training  environment  issues  addressed  earlier  in  regard  to  Table 
12. 


TABLE  13.  PS  GROUP  JUDGMENTS  RELATED  TO  FACILITATOR  FUNCTIONING 


Questions  Regarding  Facilitator  Functioning 

N/A 

Question 

(Frequency 

Strongly 

Disagree 

Response  C 
of  Student 

Disagree 

ategories 

Responses) 

Agree 

Strongly 

Agree 

16.  Answered  all  questions 

!  o 

1 

7 

7 

2 

1 7.  Maintained  control 

■■ 

0 

7 

8 

2 

1 8.  Capably  assisted  training  process 

0 

0 

6 

8 

3 

1 9.  Enforced  safety  standards 

1  3 

0 

4 

7 

3 

The  Table  13  results  suggest  considerable  variance  across  duty  stations.  They  imply  that  the 
facilitators  usually  did  an  adequate  job,  although  not  in  all  cases,  indicating  room  for 
improvement.  This  finding,  which  is  complimentaiy  to  the  training  environment  issues 
addressed  earlier  in  regard  to  Table  12,  is  particularly  meaningful.  The  training  coordinator  at 
each  duty  station  is  likely  to  play  a  pivotal  role  in  establishing  and  maintaining  the  training 
environment.  This  person’s  role  will  probably  be  exceptionally  important  to  the  acceptance  and 
effectiveness  of  nonresident  training  during  the  early  stages  of  implementation. 
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3.3.2  Comparison  of  Duty  Station  Student  Reactions  with  Other  Groups 


The  reactions  of  the  DS  group  students  were  directly  relevant  to  the  investigation  of  the 
nonresident  AN/WSC-3  training,  as  addressed  above,  representing  the  Level  1  evaluation. 
Comparison  of  the  DS  group  reactions  with  those  of  the  RM  and  RS  groups  provided  additional 
insight  into  training  implementation  issues.  Contingency  tables  presenting  the  student  response 
frequencies  for  each  question  in  the  C-School  Critiques,  for  each  group,  are  provided  in 
Appendix  E. 

Most  of  the  RM  and  RS  students  showed  positive  reactions  to  their  training,  as  exhibited  by  the 
large  majority  of  student  responses  in  the  “agree”  or  “strongly  agree”  categories.  The  percentage 
of  disagreeing  or  strongly  disagreeing  responses  for  all  questions,  for  both  groups,  ranged  from 
0%  to  33%.  Greater  than  20%  disagreement  occurred  on  only  3  questions  in  the  RM  group,  and 
on  none  in  the  RS  group.  Strong  disagreement  was  expressed  on  only  2  of  610  responses  (0.3%) 
of  the  RM  group,  and  on  4  of  363  responses  (1%)  of  the  RS  group.  Hence,  it  can  be  concluded 
that  the  two  variations  of  ANAVSC-3  training  provided  at  the  training  center  (i.e.,  CBT  and 
traditional  instructor-led)  were  received  by  the  students  at  least  as  positively  as  that  provided  at 
the  duty  stations. 


The  DS  group  reactions  were  contrasted  with  those  of  the  RM  and  RS  groups,  by  comparing  the 
distribution  of  responses  between  groups  on  similar  critique  questions  (i.e.,  similar  Likert-type 
questions).  The  questions  analyzed  were  those  addressed  earlier  regarding  the  DS  group  data  in 
Tables  9-13.  Note,  some  of  the  critique  questions  differed  for  each  group,  in  accordance  with 
the  respective  training  methods;  hence,  the  three  groups  were  not  compared  on  all  questions. 
Statistical  analyses  found  significant  differences  in  response  patterns  on  10  similar  questions, 
between  the  DS  group  and  the  RM  and/or  RS  groups.  Table  14  presents  the  questions  for  which 
significant  differences  were  found  between  groups,  and  the  response  distributions  for  each  group. 
The  questions  in  this  table  were  arranged  in  accordance  with  their  discussion  in  the  previous 
subsection  (3.3.1  Reaction  to  Duty  Station  Training),  including  the  question  groupings  addressed 
in  Tables  9, 10,  1 1  and  13  (No  significant  differences  were  found  on  questions  from  Table  12). 


The  Chi-Square  analysis  results  obtained  for  the  questions  in  Table  14  were  listed  in  the  order 
that  they  appear  in  Table  13  (Note,  the  category  “N/A”  data  were  included  in  these  analyses): 


Question  #28  -  DS,  RM:  (X2  [1]  =  7.15,  p=0.008) 

Question  #3  -  DS,  RM:  (X2  [1]  =  3.88,  p=0.049);  DS,  RS:  (X2  [1]  =  9.94,  p=0.002 
Question  #20  -  DS,  RM:  (X2  [1]  =  4.45,  p=0.035);  DS,  RS:  (X2  [1]  =  4.26,  p=0.039) 

Question  #22  -  DS,  RM:  (X2  [1]  =  6.81,  p=0.009) 

Question  #16  -  DS,  RM:  (X2  [1]  =  4.29,  p=0.038);  DS,  RS:  (X2  [1]  =  6.87,  p-0.009) 

Question  #17  -  DS,  RM:  (X2  [1]  =  13.48,  p=0.000);  DS,  RS:  (X2  [1]  =  6.05,  p=0.014); 

RM,  RS:  (X2  [1]  =  4.00,  p=0.045) 

Question  #18  -  DS,  RS:  (X2  [1]  =  5.74,  p=0.017) 

Question  #19  -  DS,  RS:  (X2  [1]  =  4.36,  p=0.037) 


32 


TABLE  14.  COMPARISON  OF  DS  WITH  RM  AND  RS  GROUPS,  ON  QUESTIONS 
WITH  DIFFERENCES _ _ _ 


TT  1 1  XX  1/11  X  Uivui 

Questions 

Student 

Group 

N/A 

Question 

(Frequency 

Strongly 

Disagree 

Response  C 
of  Student 

Disagree 

ategories 

Responses) 

Agree 

Strongly 

Agree 

F.ffect-iveness  of  AN/WSC-3  Training  (Table  9)  _ _ _ _ _ _ _ _ _ _ 

28  Course  as  effective  as  traditional  course 

DS 

0 

1 

9 

5 

2 

RM 

0 

1 

0 

10 

5 

Desitm  and  Imolementation  of  the  CBT  Course  (Table  10) 

3.  Practice  was  sufficient 

DS 

0 

t 

10 

6 

0 

RM 

0 

0 

5 

11 

0 

*  RS 

0 

0 

1 

9 

2 

Training  Environment  (Table  11)  _ _ _ _ _ _ _ _ _ 

20.  Training  environment  was  functional 

DS 

1 

2 

2 

9 

3 

RM 

0 

0 

0 

10 

6 

*  RS 

0 

0 

0 

7 

5 

_ 

22.  No  distractions  during  training 

DS 

0 

2 

5 

7 

3 

RM 

0 

0 

0 

9 

7 

Facilitator  Functioning  (Table  13)  _ _ _ _ _ _ . _ 

1 6.  Answered  all  questions 

DS 

0 

1 

7 

7 

2 

RM 

0 

0 

3 

7 

6 

RS 

0 

0 

0 

9 

4 

_ 

17.  Maintained  control 

DS 

0 

0 

7 

8 

2 

RM 

0 

0 

0 

5 

11 

RS 

0 

0 

0 

9 

4 

_ _ 

1 8.  Capably  assisted  training  process 

DS 

0 

0 

6 

8 

3 

RS 

0 

0 

0 

7 

6 

_ 

1 9.  Enforced  safety  standards 

DS 

3 

0 

4 

7 

3 

RS 

1 

0 

0 

6 

6 

Note:  *  denotes  one  or  two  students  failed  to  answer  question 


The  DS  group,  on  each  of  the  above  questions,  had  a  lower  proportion  of  agree  and/or  strongly 
agree  responses  than  did  the  comparative  group(s).  A  significant  difference  was  also  found 
between  the  RM  and  RS  groups  on  only  one  question,  #17. 

Caution  should  be  exercised  when  interpreting  these  statistical  results,  since  the  statistical 
method  assumed  only  a  single  test  was  performed,  although  a  relatively  large  number  of  tests 
were  performed  (i.e.,  one  test  for  each  question  in  the  C-School  Critique).  Consequently,  a 
heightened  probability  of  finding  statistically  significant  group  differences  existed  for  these 
analyses.  If  a  conservative  correction  factor  was  applied  to  all  results,  the  only  significant 
difference  found  at  the  0.05  level  would  have  occurred  on  question  #17,  between  the  DS  and  RM 
groups.  On  the  other  hand,  the  relatively  large  number  of  significant  differences  found  on  the 
sets  of  related  questions  suggest  a  consistent  underlying  effect  may  have  been  present,  as 
discussed  below. 
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A  common  thread  appears  through  most  of  the  questions  in  Table  14.  All,  but  questions  #28  and 
3,  directly  or  indirectly  pertain  to  the  duty  station  training  environment.  Questions  #20  and  22 
explicitly  address  the  training  environment.  Questions  #16,  #17,  #18  and  #19  address  the  CBT 
facilitator  in  the  DS  and  RM  groups,  and  the  instructor  in  the  RS  group;  these  persons  were 
certainly  an  integral  part  of  the  training  environment  for  each  group  of  students.  The  aggergate 
of  results  suggests  that,  although  the  duty  station  training  was  generally  considered  effective  and 
acceptable  by  the  DS  students,  the  training  center  environment  (i.e.,  TRACEN  Petaluma) 
resulted  in  a  higher  proportion  of  positive  student  reactions.  It  appears  that  the  facilitator  (and 
instructor)  had  a  substantial  impact  on  the  students’  reactions  to  the  respective  courses. 
Discussion  with  some  of  the  duty  station  students  following  the  testing  at  TRACEN  Petaluma,  as 
well  as  comments  made  in  the  C-School  critiques,  indicated  that  students  had  widely  differing 
experiences  interacting  with  their  facilitator.  The  duty  station  facilitator,  therefore,  appears  to  be 
an  important  element  that  may  substantially  affect  the  students’  reaction  to  duty  station  training. 
Particular  care  must  be  exercised  in  the  development  of  facilitator  skills,  and  the  execution  of  the 
facilitator-type  functions  in  support  of  duty  station  training. 

The  DS  and  RM  groups  differed  significantly  in  their  judgement  of  the  CBT  course  in 
compariosn  with  traditional  courses  (question  #28).  A  high  proportion  of  RM  group  students 
judged  the  CBT  course  as  equivalent  to  a  traditional  course  (i.e.,  15  of  16  student  responses 
agreed  or  strongly  agreed,  or  94%),  while  only  41%  of  the  DS  group  students  expressed  similar 
judgements  (as  reported  earlier).  The  significant  difference  between  these  groups  is  suggested  to 
be  due  to  the  training  environment,  since  1)  the  groups  differed  significantly  in  their  reactions  to 
their  respective  training  environments  (i.e.,  questions  #20,  #22,  #16  and  #17),  and  2)  the  CBT 
courses  were  identical  for  both  groups.  The  training  environment  was  the  only  major  factor  that 
differed  between  the  two  groups. 

In  addition,  the  duty  station  facilitators  were,  in  many  instances,  the  students’  immediate 
supervisors.  It  should  be  expected  that  these  supervisors  may  have  been  more  concerned  with 
operational  issues  than  with  training  and  experiment  issues,  during  the  period  of  AN/WSC-3 
training.  The  RM  group  facilitator,  on  the  other  hand,  was  a  TRACEN  Petaluma  staff  person 
dedicated  to  the  training.  The  RM  group  facilitator  would,  therefore,  be  expected  to  have  had  an 
attitude  more  in  accordance  with  training  concerns  during  the  AN/WSC-3  course,  as  would  the 
resident  course  instructor.  The  importance  of  command  support  of  training  at  all  levels  within 
the  command,  including  that  of  the  immediate  supervisor,  is  likely  to  be  essential  to  the  conduct 
of  effective  training. 

The  differences  between  the  DS  and  RM  groups  regarding  sufficient  practice  (question  #3)  is 
attributed  to  training  environment  differences,  following  a  rationale  similar  to  that  for  question 
#28,  since  the  practice  problems  were  identical  for  both  groups.  Despite  these  group  differences, 
a  clear  need  was  identified  for  additional  practice  problems  in  the  CBT  course.  The  DS  group 
clearly  identified  this  need,  as  discussed  earlier  (Section  3.3. 1.2,  AN/WSC-3  CBT  Course 
Content).  The  RM  group  also  identified  this  need,  notwithstanding  their  significantly  less 
disagreeing  response  to  question  #3.  The  5  “disagree”  responses  to  question  #3  by  the  RM 
group  was  the  highest  number  of  disagree  or  strongly  disagree  responses  by  these  students  to  any 
question  (Note,  question  #7  also  had  5  disagree  or  strongly  disagree  responses).  The  significant 
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differences  between  the  DS  and  RS  groups  in  this  regard,  further  indicates  the  need  for  additional 
practice  problems  in  the  AN/WSC-3  CBT  course. 

3.3.3  Application  of  Nonresident  CBT 

The  DS  and  RM  group  students  provided  suggestions  pertaining  to  the  conduct  of  training  at  duty 
stations,  in  their  responses  to  the  open-ended  questions,  and  in  their  general  remarks.  The 
critique  results  reported  in  preceding  subsections  identify  many  of  the  student  observations  and 
suggestions  for  improvement.  The  results  reported  in  this  subsection  augment  those  previously 
reported.  Note,  some  students  provided  multiple  response  to  a  question,  all  of  which  were 
included  in  the  analysis.  Hence,  the  total  number  of  responses,  in  some  instances,  exceeded  the 
number  of  students  responding. 

Thirteen  of  the  DS  group  students  responded  to  question  #48,  requesting  additional  AN/WSC-3 
topics  for  inclusion  in  the  course.  Nine  RM  students  provided  responses.  Four  DS  students  and 
one  RM  student  said  no  additional  topics  were  necessary;  the  course  was  considered  complete. 
Suggestions  from  the  remaining  students,  combined  across  the  DS  and  RM  groups,  were: 

•  Five  suggestions  identified  additional  theory,  such  as  signal  flow,  antenna  theory  and 
logic  circuits. 

•  The  remaining  suggestions  identified  other  topics,  such  as: 

Test  equipment 
Planned  Maintenance 
Crypto 
Remote  head 
Secure  communications 
AN/WSC-3  variants 
Parts  locator  function 
SATCOM 
Alignment 

Fourteen  DS  group  students  provided  over  two  dozen  recommendations  in  response  to  question 
#49,  asking  for  suggestions  of  other  USCG  training  areas  or  courses  in  which  CBT  might  be 
effective.  Twelve  RM  group  students  provided  over  a  dozen  recommendations.  The  suggestions 
included  specific  equipment,  as  well  as  general  areas  of  skill  and  knowledge.  They  were, 
combined  across  the  DS  and  RM  groups: 

•  Specific  equipment  identified  for  application  of  nonresident  training  were: 

AN/SPG69  radar 
GLH-100  Loud  Hailer 
GSB-900  transceivers 
MCX-1000 
DF 

HFDL 

Standard  Workstation  3 
Cabling  and  fiber  optics 

•  General  areas  identified  for  application  of  nonresident  training  were: 
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A-School  (completely  replace) 

SysOp  School 

Introduction  to  all  communications  equipment 

Navigation  rules 

Refresher  training 

Basic  computer  training 

General  electronic  fundamentals 

Any  equipment  at  the  Unit 

Most  equipment  on  a  small  boat 

Simple  hardware 

Most  courses 

All  correspondence  courses  (replacement  for) 

3.3.4  Summary  of  Student  Reaction  Findings 

The  students  reacted  in  a  generally  positive  fashion  to  the  nonresident  ANAVSC-3  CBT  course 
(DS  group),  as  well  as  to  the  training  center  application  of  the  CBT  course  (RM  group)  and  the 
traditional  instructor-led  resident  course  (RS  group).  The  CBT  course  design  was  judged 
positively,  including  the  general  experiment-training  interface  at  the  duty  station.  Certain  areas 
of  needed  improvement  to  the  CBT  course  were  identified  (e.g.,  providing  additional  practice 
problems),  as  well  as  improvements  to  the  duty  station  training  environment  (e.g.,  need  to  reduce 
interruptions  of  the  student).  Command  support  was  rated  positively,  as  was  facilitator 
functioning.  However,  a  relatively  wide  range  of  student  reactions  was  present  in  the  questions- 
responses  pertaining  to  the  facilitator,  suggesting  that  the  facilitators  often  did  an  adequate  job, 
although  not  in  all  cases.  The  critique  data  from  the  two  resident  training  groups  (RM  and  RS) 
suggested  student  reactions  similar  to  those  of  the  DS  group,  although  appearing  more  positive. 
Certain  findings  further  suggested  that  the  training  center  student  groups  had  a  more  positive 
reaction  to  the  training  due  to  the  training  center  environment,  in  comparison  with  the  duty 
station  environment.  The  establishment  and  execution  of  a  duty  station  environment  conducive 
to  training  was,  therefore,  identified  as  an  important  implementation  concern. 

3.4  FACILITATOR  JUDGMENTS  ABOUT  THE  ANAVSC-3  CBT  COURSE 

This  section  addresses  the  duty  station  facilitators’  reactions  to  the  ANAVSC-3  CBT  course,  and 
to  nonresident  training  overall.  Fourteen  duty  station  facilitators  provided  responses  to  all 
questions  in  the  Facilitator  Questionnaire,  excepting  question  #15,  for  which  13  responses  were 
received.  The  response  frequencies  for  the  Likert-type  questions  are  listed  in  Table  E-4  of 
Appendix  E.  Selective  results  are  presented  and  discussed  in  this  section. 

The  primary  purpose  of  the  Facilitator  Questionnaire  was  as  a  check  on  the  training-experiment 
process  conducted  at  the  duty  stations,  in  the  unlikely  event  of  major  problems  with  the 
implementation.  Consequently,  the  questions  addressed  issues  related  to  the  implementation. 

The  student  data,  as  observed  in  the  preceding  sections,  showed  that  the  duty  station 
implementation  of  this  experiment  was  satisfactory.  The  facilitator  questionnaire  data  were 
similar,  in  that  the  majority  of  responses  agreed  or  strongly  agreed  with  the  question  statements. 
Only  14%  or  less  of  the  responses  disagreed  on  any  statement;  and,  only  1  response  (of  209  total 
responses)  strongly  disagreed  with  any  statement. 
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The  level  of  command  support  and  facilitator  support  for  the  nonresident  training  were  the  issues 
of  greatest  interest  addressed  in  the  questionnaire.  The  facilitators  generally  agreed  with  the 
students  that  their  command  supported  the  ANAVSC-3  CBT  course,  as  evidenced  by  the  high 
proportion  of  agree  and  strongly  agree  responses  to  question  #1  (i.e.,  13  of  14  responses,  or 
93%).  The  facilitators  judged  their  own  support  for  CBT  as  very  positive,  in  response  to 
question  #14  (i.e.,  9  of  14  responses  were  in  the  “strongly  agree”  category;  4  were  in  the  “agree” 
category;  and  only  1  was  in  the  “disagree”  category).  These  reactions  coincided  with  those  of  the 
students,  providing  strong  support  for  nonresident  training.  The  facilitators  cited  a  variety  of 
potential  benefits  to  accrue  from  training  at  duty  stations,  in  their  responses  to  question  #16: 

•  Convenience  for  the  duty  station  was  cited  by  most  of  the  facilitators.  This  included  the 
student  being  trained  nearby  in  case  of  an  emergency;  ability  to  train  without  adversely 
affecting  the  unit’s  operational  effectiveness;  ease  of  scheduling  (i.e.,  in  comparison  with 
fitting  a  student  into  the  limited  available  course  slots);  reduction  in  time  away  for 
training;  and  the  duty  station  workload  reduction,  since  the  student  can  still  perform  work 
during  the  period  of  training,  at  a  reduced  level. 

•  Other  responses  cited  by  one  or  more  facilitators  included: 

Self-paced  training  benefits  for  the  student 
Advantages  of  computer  interaction  for  troubleshooting 
Cost  savings,  particularly  travel  costs 
CBT  stimulation  of  student  interest 
Comfortable  training  environment 

The  general  arrangements,  including  support  materials,  for  the  conduct  of  the  training- 
experiment  at  the  duty  stations  were  judged  highly  by  the  facilitators,  as  shown  in  their  responses 
to  the  four  questions  presented  in  Table  15.  All  facilitator  responses  were  in  the  “agree”  or 
“strongly  agree”  categories,  denoting  high  agreement  across  facilitators  (one  response  was  in  the 
“N/A”  category).  The  arrangements  and  materials  supporting  the  CBT  course  and  experiment 
were,  therefore,  well-received  by  the  facilitators. 


TABLE  15.  FACILITATOR  JUDGMENTS  RELATED  TO  GENERAL  TRAINING- 
EXPERIMENT  ARRANGEMENTS _ _ _ _ 


Questions  Regarding  General  Arrangements 

Question  Response  Categories 
(Frequency  of  Student  Responses) 

N/A 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

2.  Materials  satisfactorily  received 

0 

0 

0 

5 

9 

3.  Received  materials  were  sufficient 

0 

0 

0 

5 

9 

4.  Facilitator  guidance  was  satisfactory 

0 

0 

9 

5 

1 5.  Student  travel  arrangements  were  satisfactory 

r« 

0 

0 

6 

7 

Facilitator  reactions  to  the  training-experiment  process  conducted  at  the  duty  stations  are 
presented  by  their  responses  to  the  six  questions  in  Table  16.  The  overall  reactions  were 
positive,  as  indicated  by  the  high  proportion  of  responses  agreeing  with  the  question  statements 
(i.e.,  76  of  84  responses  were  in  the  “agree”  or  “strongly  agree”  categories,  or  90%).  Several 
facilitators  did,  however,  indicate  some  problems  by  their  reactions,  especially  regarding  the 
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scheduling  of  training  and  the  presence  of  interferrence  during  training  sessions  (questions  #6 
and  #8).  These  are  addressed  further  below,  in  terms  of  the  facilitators’  responses  to  the  open- 
ended  questions.  Nevertheless,  most  facilitators  did  report  a  satisfactory  experiment-training 
process. 


TABLE  16.  FACILITATOR  JUDGMENTS  RELATED  TO  TRAINING-EXPERIMENT 
PROCESS 


Questions  Regarding  Training-Experiment  Process 

N/A 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

5.  Facilitation  task  performed  satisfactorily 

0 

0 

2 

8 

4 

6.  Scheduling  of  training  performed  satisfactorily 

0 

1 

1 

8 

4 

7.  Finding  training  space  was  not  a  problem 

0 

0 

1 

7 

6 

8.  Training  sessions  ran  smoothly 

0 

0 

1 

7 

6 

9.  Little  interference  during  training 

0 

0 

2 

6 

6 

10.  Interaction  with  student  was  satisfactory 

0 

0 

0 

8 

6 

Open-ended  questions  #17,  #20  and  #21  requested  judgments  about  problems  and  how  to 
improve  training  at  duty  stations.  Several  responses  were  cited  multiple  times,  while  others  were 
unique.  The  responses  were: 

•  Scheduling  of  training  was  cited  by  several  facilitators  as  an  issue  critical  to  the  success 
of  duty  station  training.  They  included  the  following: 

The  command  must  provide  the  student  sufficient  time  to  complete  the  training. 

-  Training  and  work  task  scheduling  should  be  integrated.  Training  should  be 
scheduled  like  any  other  task  during  the  normal  work  day. 

The  unit  should  have  flexibility  to  schedule  training  in  accordance  with  operational 
demands. 

•  Training  disruptions  (i.e.,  interferring  with  the  student’s  training  process)  was  cited  by 
several  facilitators  as  a  potential  problem.  The  training  environment  must  be  free  from 
distractions.  Students  also  identified  this  as  a  problem,  several  having  experienced 
disruptions  during  their  CBT  course. 

•  Additional  responses  identifying  potential  problems  and  suggestions  for  improvement 
were: 

-  Establish  a  permanent  facilitator  at  each  duty  station  (cited  by  several  facilitators). 
One  response  suggested  eliminating  the  facilitator. 

-  Have  all  CBT  hardware  and  software  located  permanently  at  units  (cited  by  several 
facilitators). 

-  Testing:  Automate  testing;  perform  testing  at  each  unit;  have  USCG  Institute  perform 
all  testing. 

Have  dedicated  spaces  for  training  at  each  unit. 

-  Give  facilitator  longer  lead  time  before  the  start  of  the  course. 

-  Need  to  have  procedures  to  secure  hardware  and  software  from  potential  theft. 

-  Need  scheduling  of  sufficient  facilitator  time  (similar  to  student  time,  above). 

Student  motivation  may  be  a  problem 
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The  quality  of  the  CBT  course  was  judged  highly  by  the  facilitators,  as  shown  by  their  responses 
to  the  questions  in  Table  1 7.  All  but  one  response  agreed  or  strongly  agreed  with  the  statements, 
showing  substantial  concurrence  among  the  facilitators.  The  facilitator  reactions  about  the 
course  quality  agree  well  with  those  of  the  students. 


TABLE  17.  FACILITATOR  JUDGMENTS  RELATED  TO  AN/WSC-3  CBT  COURSE 


Questions  Regarding  Course  Quality 

(i 

N/A 

Question 
frequency  o 

Strongly 

Disagree 

Response  ( 
f  Facilitate 

Disagre 

e 

Categories 
>r  Respond 

Agree 

ses) 

Strongly 

Agree 

1 1 .  Self-oaced  method  was  convenient 

0 

0 

1 

6 

7 

12.  CBT  media  quality  was  satisfactory 

0 

0 

0 

6 

8 

13.  Computer  &  software  worked  satisfactorily 

0 

0 

0 

4 

10 

The  facilitators  provided  a  variety  of  suggestions  for  improving  the  AN/WSC-3  CBT  course  (i.e., 
questions  #18  and  #19),  such  as: 

•  Elimation  of  the  technical  manuals. 

•  Mandatory  use  of  the  maintenance  log  sheets  during  the  course. 

•  Providing  additional  clarifications  to  the  CBT  content. 

•  Editorial  and  logisitics  suggestions  for  the  Facilitator  Guidelines. 

In  summary,  the  facilitators’  reactions  coincided  well  with  those  of  the  students.  The  facilitators 
were  positive  about  the  CBT  course  conducted  at  their  duty  stations,  indicating  generally  high 
approval  of  the  arrangements,  materials  and  course  content.  They  indicated  that  the  course  was 
supported  by  their  command,  and  that  nonresident  training  has  good  potential.  They  also 
indicated  several  areas  that  must  be  carefully  addressed  to  assure  nonresident  training  is 
effectively  implemented. 

4  TRAINING  EFFECTIVENESS  DISCUSSION 

This  section  addresses  the  combined  results,  as  they  pertain  to  the  effectiveness  of  nonresident 
CBT  at  duty  stations.  Data  were  collected  using  a  variety  of  measures,  as  reported  earlier.  The 
consistency  of  results  across  these  measures,  as  well  as  the  results  pertaining  to  inter-evaluator 
assessments,  indicates  good  reliability  in  the  data  and  confidence  in  the  results. 

4.1  TRAINING  GAIN 

Each  of  the  training  methods  were  effective  in  student  achievement  of  AN/WSC-3  knowledge,  as 
demonstrated  by  significant  gains  between  the  AN/WSC-3  part  of  the  Pretest  and  that  of  the 
Posttest  (See  Table  8).  The  absence  of  a  corresponding  significant  increase  in  scores  on  the 
general  part  of  the  tests  suggests  that  the  courses  were  highly  focused  on  AN/WSC-3  content. 
These  findings,  alone,  indicate  that  the  three  training  methods  were  effective.  And,  most 
importantly,  these  findings  demonstrate  the  potential  for  nonresident  training  in  the  USCG. 
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4.2  EFFECTIVENESS  OF  THE  TRAINING  METHODS 


The  preceding  subsection  identified  the  achievement  of  training  gain  for  each  training  method, 
and  their  training  effectiveness.  This  subsection  addresses  similarities  and  differences  in  training 
effectiveness  between  the  three  training  delivery  methods. 

4.2.1  Hands-On  Test  Problems 


Student  performance  in  solving  the  five  malfunctioning  AN/WSC-3  hands-on  problems  was 
considered  the  single  most  important  measure  relating  to  the  effectiveness  of  the  training  at  duty 
stations.  This  test  replicated,  to  a  degree,  the  types  of  problems  and  working  environment  ETs 
would  encounter  in  the  field.  Each  hands-on  problem  began  with  a  simple  statement  of  the 
trouble-symptom  and  current  situation  at  a  duty  station  (problem  statements  are  presented  in 
Appendix  A).  The  student  was  left  alone  to  troubleshoot  and  repair  the  AN/WSC-3(v)7  UHF 
transceiver,  using  available  test  equipment,  technical  manuals,  and  other  resources  that  would 
typically  be  available  at  the  duty  station.  Several  students  later  commented  that  the  test 
environment  was  very  similar  to  that  of  a  duty  station,  and  unlike  a  typical  training  environment. 
This  test  was  viewed  as  a  bottom-line  measure  of  the  student’s  proficiency  in  solving  applied 
AN/WSC-3  maintenance  problems.  The  highly  structured  evaluation  process,  together  with  the 
two  independent  evaluators,  added  to  the  relative  importance  of  this  measure. 

The  Resident  group  (RS)  was  used  as  the  comparative  baseline  for  evaluation  of  the  CBT  course, 
which  included  both  experimental  delivery  contexts  (i.e.,  duty  station  and  training  center 
environments).  As  noted  earlier,  the  instructor-led  resident  course  was  considered  an  effective 
USCG  course.  The  student  performance  reinforced  that  opinion,  finding  a  large  significant 
training  gain  to  result  from  the  resident  course.  The  AN/WSC-3  CBT  course  development  goal 
was  to  achieve  a  nonresident  version  of  the  AN/WSC-3  course  that  was  equivalent  to  the 
instructor-led  resident  version. 

The  mean  hands-on  problem  scores  achieved  by  students  in  each  group,  across  the  five  problems, 
are  shown  in  Figure  2.  The  statistical  analysis  found  that  the  CBT  Remote  (RM)  group 
performed  significantly  better  than  the  RS  group.  On  the  other  hand,  no  significant  differences 
were  found  between  the  CBT  Duty  Station  (DS)  and  RM  groups,  nor  between  the  DS  and  RS 
groups.  Furthermore,  inspection  of  Figure  2  shows  that  the  mean  scores  between  all  groups  were 
very  close.  The  RM  group  achieved  the  highest  score,  but  only  by  a  very  small  margin  above  the 
DS  group  (i.e.,  much  less  than  1%).  And,  the  difference  between  these  two  CBT  groups  and  the 
RS  group  was  also  small  (i.e.,  about  4%).  Finally,  each  group  achieved  the  highest  mean  score 
on  at  least  one  of  the  five  problems  (See  Table  5).  These  collective  results  suggest  little 
meaningful  difference  between  the  training  methods  (i.e.,  approximate  parity  of  the  training 
methods).  That  is,  for  practical  purposes,  the  three  training  methods  were  approximately 
equivalent  in  achieving  AN/WSC-3  qualified  ETs.  These  results  demonstrate  that  a  Coast  Guard 
non-resident  CBT  operation  and  maintenance  course  taught  at  duty  stations  can  be  equal  in 
effectiveness  to  a  traditional  resident  course  taught  at  a  training  center. 
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FIGURE  2.  HANDS-ON  TEST  MEAN  SCORES  FOR  EACH  GROUP 


This  important  finding  was  supported  by  the  consistency  obtained  between  the  hands-on  problem 
scores  (Figure  2)  and  the  evaluators’  summary  judgments  about  performance  during  each  hands- 
on  problem.  Figure  3  shows  the  mean  Overall  Rating  awarded  by  the  evaluators  for  each  training 
method  group.  The  order  of  these  overall  ratings  across  the  groups  agree  with  that  of  the  student 
performance  scores  cited  above  (i.e.,  RM  group,  followed  by  DS  group,  and  then  by  RS  group). 
This  consistency  between  judgments  at  different  levels  of  evaluation  attests  to  the  reliability  of 
the  training  effectiveness  results. 


FIGURE  3.  OVERALL  PROFICIENCY  RATINGS  OF  STUDENTS  DURING  HANDS- 
ON  PROBLEMS 
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The  evaluator  summary  judgments  of  the  other  student  attributes  were  similarly  consistent  across 
groups,  as  shown  in  Figure  4.  The  relative  differences  between  groups  were  the  same  for  each  of 
the  other  four  attributes  (i.e.,  RM  group,  followed  by  DS  group,  and  then  followed  by  RS  group). 
Furthermore,  the  mean  rating  differences  between  groups  for  each  attribute  were  relatively  small, 
resulting  in  the  finding  of  no  significant  group  differences  on  any  of  the  attributes. 
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FIGURE  4.  OTHER  ATTRIBUTE  RATINGS  OF  STUDENTS  DURING  HANDS-ON 
PROBLEMS 

The  multiple  results  of  student  performance  during  the  hands-on  problems  clearly  demonstrate 
that  nonresident  CBT  methods  can  be  as  effective  as  traditional  resident  training  methods.  Based 
on  the  premise  that  the  resident  AN/WSC-3  course  was  effective,  these  findings  further  show 
that  nonresident  CBT  is  effective.  Finally,  and  most  importantly,  the  findings  show  that  training 
at  Coast  Guard  duty  stations  can  be  equally  effective  to  similar  training  at  a  training  center. 

4.2.2  Posttest 

Student  performance  on  the  Posttest,  following  completion  of  the  respective  AN/WSC-3  courses, 
showed  a  pattern  similar  to  that  of  the  hands-on  problems  (See  Figure  5).  This  time,  however, 
the  RS  group  scored  highest,  with  the  DS  group  in  the  middle,  and  the  RM  group  achieving  the 
lowest  mean  Posttest  score.  This  pattern  was  the  same  for  the  General  and  AN/WSC-3  parts  of 
the  Posttest.  Importantly,  the  three  groups  again  differed  by  relatively  small  amounts  on  each  of 
these  parts.  The  statistical  analyses  did  not  find  significant  differences  between  the  three  groups, 
on  either  the  AN/WSC-3  part  or  the  General  part. 
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The  Posttest  results,  therefore,  demonstrate  findings  similar  to  those  of  the  hands-on  problems. 
The  CBT  course  was  equally  effective  to  the  traditional  resident  course.  Of  greater  importance, 
the  CBT  course  conducted  in  the  nonresident  duty  station  environment  was  equally  effective  to 
both  the  traditional  resident  course  and  the  CBT  course  conducted  in  the  training  center 
environment. 
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FIGURE  5.  POSTTEST  RAW  SCORE,  INCLUDING  GENERAL  AND  ANAVSC-3 

PARTS 


4.2.3  Problem  Solving  Time 

Similar  to  most  of  the  training  effectiveness  results,  the  mean  problem  solving  time  for  the 
hands-on  problems  did  not  differ  significantly  between  the  three  training  method  groups. 

Figure  6  shows  the  mean  time  to  solve  the  hands-on  problems  for  each  group.  The  range  of 
student  times  is  shown  by  the  brackets  around  the  mean  for  each  group  (Note,  the  maximum  time 
for  any  problem  was  90  minutes).  Substantial  time  ranges  existed  within  each  group,  on  most 
problems.  These  large  time  differences  between  students  in  each  group  negated  the  possibility  of 
significant  time  differences  between  groups.  The  problem  solving  time  results,  like  the  preceding 
student  performance  data,  indicate  approximate  equivalence  between  training  method  groups, 
and  hence  resident  and  nonresident  training  delivery. 
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4.3  SUMMARY  INTEGRATION  OF  TRAINING  EFFECTIVENESS 
FINDINGS 

The  two  sets  of  student  performance  data,  paper-and-pencil  test  and  hands-on  troubleshooting 
problems,  have  found  approximately  equivalent  student  achievement  in  each  of  the  three  training 
method  groups.  The  CBT  course  was  found  to  be  as  effective  as  the  traditional  instructor-led 
resident  course.  More  importantly,  computer-based  training  at  duty  stations  was  found  to  be 
equally  effective  to  instructor-led  training  at  a  resident  training  center.  This  finding  has 
important  implications  for  the  future  direction  of  the  USCG’s  training  system.  It  provides  the 
USCG  with  greatly  expanded  options  for  achieving  cost-effective  training  delivery. 

It  is  important  to  recognize  that  these  findings  pertain  to  the  potential  effectiveness  of  training  at 
duty  stations.  They  are  limited  in  addressing  1)  the  type  of  training  that  would  be  effective  at 
duty  stations;  and  2)  the  duty  station  conditions  necessary  to  support  effective  training.  It  is 
unlikely  that  all  current  USCG  training  could  be  effectively  conducted  as  CBT  courses  at  duty 
stations.  Nevertheless,  it  is  likely  that  a  substantive  subset  of  current  training  may  be  effectively 
accomplished  at  duty  stations. 

The  results  of  the  C-School  Critique  analysis  support  the  major  test  findings,  indicating  that  the 
nonresident  CBT  was  effective,  and  as  effective  as  the  traditional  resident  training.  However,  the 
critique  responses  also  suggest  that  student  attitudes  toward  the  nonresident  CBT,  while  positive, 
were  not  as  strongly  positive  as  those  toward  resident-based  training  (i.e.,  whether  CBT  or 
instructor-led  training  methods).  This  was  attributed  to  differences  in  the  duty  station  and 
resident  training  center  environments.  This  indicates  a  need  to  carefully  construct,  and  maintain, 
the  duty  station  environment  in  a  manner  conducive  to  the  achievement  of  training. 
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This  experiment,  and  the  duty  station  training  that  was  conducted,  was  accomplished  under 
controlled  conditions.  Although  relatively  little  direct  control  was  exerted  over  the  duty  station 
conditions  during  each  student’s  training,  the  level  of  attention  and  coordination  with  each  duty 
station  unit  certainly  helped  to  motivate  the  personnel  involved.  Even  though  some  students  did 
report  training  situation  difficulties  at  their  duty  stations,  it  is  reasonable  to  assume  that  the  CBT 
training  received  a  fair  amount  of  attention.  For  this  reason,  the  results  should  be  interpreted  as 
representing  the  potential  of  nonresident  CBT  in  the  USCG  (i.e.,  if  duty  station  training  is 
appropriately  managed,  and  supported  by  the  various  levels  within  the  USCG  infrastructure). 
Achievement  of  reliably  effective  training  at  duty  stations,  on  a  regular  production  training  basis, 
is  likely  to  require  serious  change  in  the  way  the  USCG  conducts  business  at  duty  stations.  The 
student  critique  and  implementation  analysis  interviews  identify  issues  related  to  implementing 
nonresident  training  at  Coast  Guard  duty  stations. 

4.4  TRAINING  TIME 

The  time  required  to  complete  the  ANAVSC-3  course  is  an  important  factor.  Not  for  evaluating 
the  course’s  effectiveness,  but  rather  for  evaluating  the  course’s  efficiency  and  cost.  Before 
reviewing  the  training  time  results,  it  is  important  to  note  that  the  duty  station  training  time  data 
suffered  from  the  same  control  limitations  as  the  Pretest  data  (Pretest  control  problems  noted 
earlier  in  this  report).  The  training  time  was  automatically  recorded  by  the  computer  when  the 
ANAVSC-3  CBT  course  was  in  use.  However,  the  recording  made  no  distinction  regarding 
purpose  of  use  (e.g.,  computer  left  unattended  during  operation).  Consequently,  the  obtained 
results  for  the  CBT  Duty  Station  Group  must  be  cautiously  interpreted  as  the  training 
environment  was  not  controlled.  Due  to  this  method  of  accumulating  the  duty  station  time  data, 
the  amount  of  time  recorded  was  never  an  underestimation  of  the  actual  training  time,  but  may 
have  been  an  overestimation  in  some  cases. 

There  are  three  known  instances  where  the  recorded  training  times  may  be  considered  an 
overestimation.  In  one  instance,  a  student  in  the  CBT  Duty  Station  Group  accidentally  left  the 
laptop  computer  running  and  failed  to  exit  from  the  CBT  courseware  after  completing  a  section 
of  the  course.  The  result  was  a  total  training  time  of  33:3 1  hours:minutes  recorded  for  this 
student.  The  second  and  third  instances  in  the  CBT  Duty  Station  Group  involved  a  facilitator 
who  was  surprised  to  see  how  quickly  the  two  members  of  his  unit  completed  the  CBT  portion  of 
the  course,  and  therefore  directed  the  two  members  to  go  back  through  the  courseware  one  more 
time.  All  three  of  these  recorded  training  times  were  used  in  calculating  the  arithmetic  mean 
time  for  the  CBT  Duty  Station  group  to  complete  the  CBT  portion  of  the  ANAVSC-3  course. 

Despite  the  overestimation  of  training  completion  times  for  the  CBT  Duty  Station  Group,  the 
mean  (average)  training  times  recorded  for  the  two  CBT  groups  (presented  in  Table  1 8)  are 
impressive  when  compared  to  the  completion  time  for  the  instructor-led  resident  version  of  the 
course  (also  presented  in  Table  18). 
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TABLE  18.  STUDENT  TRAINING  TIME  (HOURSrMINUTES) 


GROUP 

CBT 

COURSEWARE 

TIME 

(arithmetic  mean) 

HANDS-ON 

TRAINING 

TIME 

TOTAL  TIME 

CBT  Duty  Station 

10:56 

2:00 

12:56 

CBT  Remote 

5:49 

2:00 

7:49 

Resident 

N/A 

N/A 

The  total  time  for  the  resident  student  to  complete  the  instructor-led  AN/WSC-3  course  is  36 
hours.  In  the  instructor-led  course,  hands-on  time  on  the  actual  AN/WSC-3  UHF  transceivers 
and  time  performing  other  classroom  activities  (e.g.,  listening  to  the  instructor  lecture)  varies 
based  on  instructor  preference  and  the  ability  of  the  attending  class.  Therefore,  no  attempt  is 
made  to  separate  the  Resident  student’s  time  by  category  in  Table  18. 

The  arithmetic  mean  time  for  the  CBT  Duty  Station  student  to  complete  the  CBT  portion  of  the 
course  was  10:56  hours:minutes.  Adding  the  2-hours  required  to  complete  the  Hands-on 
Training  portion  of  the  course  to  the  arithmetic  mean  time  to  complete  the  CBT  portion  of  the 
course  (10:56  hours:minutes),  the  total  time  to  complete  the  course  is  12:56  hours:minutes. 

When  compared  to  the  time  required  to  complete  the  instructor-led  resident  version  of  this  course 
(36  hours),  the  arithmetic  mean  time  to  complete  the  course  is  reduced  by  64%  when  taking  the 
CBT  version  of  the  course  at  the  duty  station.  Of  interest,  if  the  time  recorded  for  the  student 
who  failed  to  turn-off  the  computer  after  completing  a  session  of  training  were  to  be  removed 
(33:31  hours:minutes),  the  arithmetic  mean  time  for  this  group  would  drop  to  10:09 
hours:minutes. 

The  arithmetic  mean  time  for  the  CBT  Remote  student  to  complete  the  CBT  portion  of  the  course 
is  5:49  hours  :minutes.  The  lower  completion  time  in  comparison  to  the  CBT  Duty  Station 
students  reflects  the  isolation  from  the  duty  station  environment,  and  the  closer  facilitator 
oversight  that  is  experienced  by  the  CBT  Remote  students  at  TRACEN  Petaluma.  Adding  the  2- 
hours  required  to  complete  the  Hands-on  Training  portion  of  the  course  to  the  arithmetic  mean 
time  to  complete  the  CBT  portion  of  the  course  (5:49  hours  minutes),  the  total  time  to  complete 
the  course  is  7:56  hours  minutes.  When  compared  to  the  time  required  to  complete  the 
instructor-led  resident  version  of  this  course  (36  hours),  the  arithmetic  mean  time  to  complete  the 
course  is  reduced  by  78%  when  taking  the  CBT  version  of  the  course  at  a  training  center. 

The  difference  in  course  completion  times  between  the  CBT  Remote  group  (7:56  hoursminutes) 
and  the  CBT  Duty  Station  group  (12:56  hoursminutes)  indicates  that  training  completion  times 
at  duty  stations  could  be  shortened.  Decreasing  course  completion  times  at  the  duty  station 
would  most  likely  require  increased  facilitator  oversight  and/or  the  creation  of  an  environment  at 
the  duty  station  that  is  more  conducive  to  training.  These  types  of  changes  come  at  a  cost,  and 
therefore,  a  cost-benefit  analysis  would  have  to  be  performed  to  determine  if  decreasing  the 
student’s  time  to  complete  a  nonresident  course  at  the  duty  station  is  worth  the  additional  cost. 
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DUTY  STATION  IMPLEMENTATION  ANALYSIS  FINDINGS 


This  section  presents  the  findings  of  the  implementation  analysis  interviews  conducted  at  duty 
stations.  The  findings  of  this  section: 

•  Identify  important  issues  and  potential  problems  associated  with  the  implementation  of 
nonresident  CBT  at  duty  stations. 

•  Suggest  approaches  and  considerations  for  dealing  with  many  of  the  issues  and  problems, 
to  achieve  effective  implementation  of  nonresident  CBT  at  duty  stations. 

The  issues,  problems  and  suggested  approaches  stem  directly  from  the  interviews.  Also  included 
were  information  from  discussions  with  some  duty  station  group  students  at  TRACEN  Petaluma, 
and  findings  of  the  Student  C-School  Critique.  It  should  be  recognized  that  considerable  in- 
depth  investigation  is  required  to  develop  suitable  solutions  to  many  of  the  issues  and  problems 
identified. 

It  should  be  noted  that  many  of  the  issues  identified,  and  recommendations  proposed  in  this 
section,  would  be  common  to  other  nonresident  media  considerations  (e.g.,  interactive 
video-teletraining. 

5.1  OVERALL  OPINION  OF  NONRESIDENT  CBT 

The  majority  of  persons  interviewed  showed  strong  support  for  nonresident  CBT.  They  generally 
expressed  the  belief  that  nonresident  CBT  can  and  will  work  at  duty  stations,  and  that  it  is  good 
for  the  USCG.  This  strong  support  was  present  at  all  levels  of  persons  interviewed  (e.g., 
Commanding  officers,  supervisors,  students),  whether  on  ships  or  at  shore  stations,  and  in  all 
three  of  the  major  duty  station  areas  (i.e.,  East,  Gulf  and  West  Coast  units).  Examples  of  specific 
comments  received  were: 

“Just  do  it” 

“A  real  plus  for  duty  stations” 

“Exciting”,  “good  idea”  and  “  nonresident  training  will  work” 

A  wide  variety  of  reasons  for  the  strong  positive  opinions  were  cited,  several  by  most  persons,  as 
follows: 

•  Less  adverse  impact  than  resident  training  on  the  overall  team’s  ability  to  perform  its 
mission. 

•  Unit  flexibility  -  To  meet  training  and  operating  needs.  This  was  cited  by  persons  in 
most  units  as  a  major  factor  in  support  of  nonresident  CBT.  The  opinion  was  that  major 
training  needs  are  difficult  to  meet  today,  due  to  limited  school  quotas,  course  scheduling 
conflicts,  and  limits  on  the  number  of  staff  who  can  receive  training  in  a  particular  skill 
area.  The  nonresident  courses  would  give  the  units  flexibility  in  determining  their 
training  needs,  scheduling  and  conducting  training. 

•  Assists  minimally-staffed  vessels.  Some  stated  that  nonresident  CBT  was  a  “key”  to 
success  with  minimally-staffed  vessels,  since  it  reduced  student  separation  time  and 
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overall  training  time.  It  was  also  stated,  in  this  regard,  that  nonresident  CBT  would  allow 
more  cross-training  of  staff. 

•  Assist  with  general  unit  training  requirements.  Several  units,  particularly  the  cutters, 
stated  that  they  had  a  variety  of  training  to  accomplish  with  most  new  crewmembers,  such 
as  damage  control  qualification.  A  computer-based  nonresident  training  capability,  if 
also  addressing  these  training  needs,  would  provide  substantial  assistance.  It  would 
reduce  training  time,  and  free  other  staff  at  the  unit  from  instructor  duty. 

•  Availability  of  staff  while  undergoing  training.  Staff  members  would  be  available  to 
assist  with  emergencies.  If  the  CBT  course  were  conducted  only  during  a  portion  of  the 
workday,  which  would  likely  occur  in  most  courses,  the  staff  member  would  be  available 
to  perform  a  portion  of  their  normal  workload  during  the  day. 

•  Eases  unit  staffing  problems.  Several  persons  felt  that  units  are  often  understaffed. 
Nonresident  CBT,  with  the  student  still  available  to  participate  in  some  daily  work,  would 
ease  their  staffing  problems. 

•  Minimizes  unit  disruptions.  Other  staff  members  must  fill-in  during  a  student’s  absence 
for  resident  training,  causing  general  staff  disruptions.  Nonresident  CBT  will  reduce  or 
eliminate  such  disruptions. 

•  Training  efficiency.  The  reduced  training  time  typically  associated  with  a  CBT  course 
was  viewed  as  a  net  time/resource  savings  to  the  unit.  It  was  suggested  that,  in 
comparison  with  nonresident  training  (e.g.,  CBT,  IVT,  correspondence),  resident-based 
training  is  inefficient. 

•  Training  process  benefits.  These  included: 

Self-paced  training,  enabling  learning  at  the  student’s  preferred  rate. 

Able  to  return  to  previously  learned  material. 

Refresher  training,  at  a  later  date. 

Greater  consistency  across  multiple  offerings  of  courses. 

•  Individual  student  benefits.  These  included: 

Benefit  to  families,  by  reducing  separation  time. 

Reducing  hassles  associated  with  travel  expenses  and  claims. 

Less  training  time  (i.e.,  which  is  usually  associated  with  CBT  courses). 

Only  one  individual  who  participated  in  the  CBT  Duty  Station  group,  expressed  serious  doubt 
about  nonresident  CBT.  His  doubt  was  not  based  on  an  opinion  that  nonresident  CBT  was 
fundamentally  ineffective,  but  rather  on  the  belief  that  the  USCG  policies  necessary  to  effectively 
conduct  such  training  at  duty  stations  would  not  be  realistically  implemented.  His  opinion  was 
based,  in  part,  on  his  experiences  during  the  experiment  (e.g.,  he  had  to  train  on  his  own  time,  at 
home).  Furthermore,  he  expressed  an  opinion  of  not  liking  computers.  Generally,  however,  the 
interviewed  students  were  positive  about  the  potential  of  nonresident  CBT,  contributing  to  the 
reasons  cited. 

Although  the  persons  interviewed  were  positive  about  the  potential  of  nonresident  CBT,  they 
also  identified  a  variety  of  potential  drawbacks  to  its  effective  implementation.  These  issues  are 
addressed  in  later  subsections.  The  potential  problems  represent  real  threats  to  the  successful 
implementation  of  nonresident  CBT.  Nevertheless,  it  was  the  opinion  of  most  persons 
interviewed  (i.e.,  all  persons  but  the  one  student  noted  above)  that  all  of  the  identified  problems 
can  be  overcome,  and  successful  nonresident  training  can  be  implemented.  However,  it  was  also 
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the  opinion  of  many  persons  interviewed  that  careful  and  decisive  actions  must  be  taken  by  the 
USCG  to  assure  that  these  problems  are  adequately  controlled,  or  else  the  nonresident  CBT  will 
not  be  successful. 

5.2  OPINIONS  ABOUT  THE  AN/WSC-3  CBT  COURSE 

This  subsection  presents  the  opinions  of  duty  station  staff  regarding  the  AN/WSC-3  CBT  course 
used  in  the  experiment. 

5.2.1  General  Duty  Station  Staff  Opinions 

The  duty  station  staff  interviewed  were  generally  positive  about  the  AN/WSC-3  CBT  course, 
similar  to  their  opinions  regarding  nonresident  CBT  in  general.  It  should  be  noted  that  not  all 
persons  interviewed  had  actually  observed  the  CBT  course,  although  all  were  at  least  generally 
aware  of  it.  The  comments  included  the  following  observations: 

•  AN/WSC-3  CBT  course  was  effective.  Virtually  all  persons  stated  this  opinion,  to  the 
extent  they  were  familiar  with  the  course.  The  students  stated  that  they  were  comfortable 
with  the  AN/WSC-3  (v)7  UHF,  and  the  idea  of  working  on  it,  although  not  all  students 
actually  had  performed  work  on  their  unit’s  AN/WSC-3.  At  the  other  extreme  were 
individuals  who  had  not  actually  seen  the  CBT  course,  but  had  heard  that  it  was  a  good 
course. 

•  Attitude  toward  the  AN/WSC-3  CBT  course  changed  after  exposure  to  the  course. 
Several  persons  reported  that  their  initial  thoughts  about  the  course  were  negative.  They 
expected  a  relatively  poor  course.  However,  their  attitude  changed  with  use  of  the  course, 
to  one  of  praise. 

•  Several  suggestions  for  improvement  of  the  AN/WSC-3  CBT  course  were: 

-  More  practice  problems  needed.  This  was  also  identified  in  the  C-School  Critique. 

-  Additional  detailed  notes  should  be  provided,  such  as  shortcuts  and  experience 
insights  (i.e.,  Similar  to  information  an  instructor  would  provide). 

-  Alignment  problems  were  wanted. 

-  Manuals  should  be  put  on  the  computer. 

These  observations  are  important  to  the  implementation  analysis,  since  they  represent  the  near- 
term  basis  for  individual  judgments  about  nonresident  CBT. 

5.2.2  Student  Opinions 

The  student  opinions  about  the  AN/WSC-3  course,  in  addition  to  the  reports  in  the  preceding 
subsection  and  those  regarding  the  C-School  Critique  responses  cited  earlier,  provided 
information  regarding  the  context  in  which  their  training  was  conducted.  Most  of  the  students 
interviewed  noted  that  their  AN/WSC-3  CBT  training  was  conducted  under  difficult 
circumstances.  Problems  noted  included: 
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Absent  facilitator 
Training  on  personal  time 

Training  while  on  TAD,  away  from  their  unit  to  perform  work 
Interruptions  from  various  persons,  for  a  variety  of  reasons 
Uncomfortable  space 

Generally,  students  gave  the  opinion  that  they  considered  the  duty  station  context,  as  it  was  when 
they  took  the  AN/WSC-3  CBT  course,  difficult  for  training.  Furthermore,  it  was  their  opinion 
that,  for  nonresident  CBT  to  succeed,  the  duty  station  context  must  be  changed  to  be  supportive 
of  training  activities.  These  issues,  and  others,  are  addressed  further  in  later  subsections. 

5.3  TRAINING  INTERFERENCE.  TIME  AND  SURROUNDINGS 

The  duty  station  environment  in  which  training  occurs  can  have  a  substantial  impact  on  the 
effectiveness  and  efficiency  of  the  training  process.  Three  components  of  the  training 
environment  were  identified  as  potentially  major  problems: 

•  Interference  with  the  student  during  a  training  session. 

•  Time  available  for  conduct  of  the  student’s  training  session. 

•  Disruptions  due  to  the  surroundings  (i.e.,  location  and  space  in  which 
training  is  conducted). 

These  three  components  are  closely  related,  in  terms  of  the  potential  problems,  as  well  as  their 
potential  solutions.  Each  problem  is  addressed  in  a  separate  subsection,  bearing  in  mind  that  they 
overlap.  The  potential  solutions  are  addressed  together  in  the  final  subsection,  integrated  across 
the  problems. 


5.3.1  Interference  Problem 

Interference  with  a  student  who  is  actively  engaged  in  a  training  session  is  considered  a 
potentially  major  problem  facing  the  implementation  of  CBT  at  duty  stations.  This  expected 
problem  was  identified  by  most  of  the  persons  interviewed.  The  potential  adverse  impact  of 
interference  is  multi-faceted,  including: 

•  Reduced  training  effectiveness,  to  the  extent  that  course  objectives  are  not 
achieved. 

•  Reduced  training  efficiency,  requiring  substantially  increased  time  to 
complete  the  course. 

•  Reduced  student  comfort  and  heightened  anxiety,  stemming  from  their 
concerns  regarding  operational  issues.  This  can  be  very  disruptive  to  their 
training  process. 

•  Reduced  student  motivation,  which  can  adversely  impact  the  training 
effectiveness  and  efficiency. 
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Training  interruptions  may  occur  in  many  forms  such  as:  a  fellow  worker  asking  a  brief 
technical  question;  noise  onboard  the  ship;  pressure  from  a  supervisor  to  finish  the  training 
session  quickly  so  the  student  can  get  back  to  the  work  task;  and,  premature  termination  of  the 
training  session  because  the  student  is  being  sent  on  an  urgent  repair  mission,  as  a  result  of 
direction  from  the  district  command. 

Real  emergencies  should  take  precedence  over  training,  and  interrupt  the  training  process.  But, 
these  should  be  infrequent  events,  which  have  a  minimal  impact  on  training  over  the  long  term. 

Most  students  interviewed  noted  they  had  interruptions  when  training  at  the  duty  station.  Several 
students  stated  that  they  went  home  to  train,  primarily  to  avoid  the  continuing  interruptions. 

Some  students  stated  that  interruptions  were  minor;  these  were  in  a  more-controlled  training 
environment. 

Most  persons  felt  that,  although  interruptions  are  a  potential  problem,  they  can  be  adequately 
dealt  with. 


5.3.2  Time  Problem 

The  time  available  for  training  was  considered  by  the  students  to  present  a  potential  problem.  At 
least  one  student,  apparently,  was  told  to  train  on  his  own  time.  This  approach,  obviously,  will 
not  work  if  a  substantial  training  load  is  transferred  to  duty  stations. 

Most  units  are  busy,  using  all  available  personnel  to  perform  required  work.  Training,  almost 
always,  is  viewed  as  a  secondary  activity,  one  that  can  be  easily  postponed  to  a  less  demanding 
time  period.  Although  this  approach  may  work  in  today’s  context,  in  which  students  are 
removed  from  their  duty  stations  to  training  centers  for  serious  training,  it  will  not  work  when  a 
large  amount  of  necessary  training  is  conducted  at  the  duty  stations. 

The  supervisory  and  command  personnel  generally  felt  that  the  time  available  for  training, 
although  a  potential  problem  since  operational  needs  almost  always  take  precedent  over  training 
needs,  can  be  effectively  managed.  Their  primary  rationale  was  that  nonresident  CBT  should 
provide  a  net  gain  to  the  unit,  of  available  personnel  work  hours.  That  is,  the  bottom-line  of 
nonresident  CBT  should  be  less  student  training  hours,  and  more  personnel  work  hours.  This 
was  viewed  as  a  benefit  of  the  travel  time  savings,  and  the  typical  training  time  reduction 
associated  with  CBT  courses. 

5.3.3  Training  Surroundings  Problem 

The  training  surroundings  are  comprised  of  the  location  in  which  the  duty  station  training  would 
take  place,  such  as  an  office  or  a  compartment  on  a  cutter.  They  involve  the  physical  space,  as 
well  as  other  aspects  of  the  surroundings,  such  as  noise  and  vibration.  The  surroundings  can  be 
conducive,  or  disruptive,  to  training.  The  interruption  problem  identified  above,  for  example, 
can  be  greatly  affected  by  the  chosen  training  location.  The  interruptions  cited  by  the 
interviewed  students  were  largely  due  to  the  location  chosen  for  their  training. 
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The  shore  units  have  some  potential  surroundings  problems.  They  are  generally  associated  with 
disruptive  noise  from  nearby  machines  and  activities  (e.g.,  hammering,  noisy  machinery),  and 
people  talking.  These  problems,  although  potentially  great,  should  be  relatively  easy  to  solve, 
such  as  by  locating  students  in  an  appropriate  office  space.  Some  persons  indicated  their  unit  had 
appropriate  spaces  to  accommodate  students,  while  others  said  they  were  short  on  space.  At  least 
two  of  the  AN/WSC-3  CBT  students  were  initially  positioned  in  a  machinery  shop  area  for  the 
training,  alongside  other  workers  performing  normal  maintenance  work.  This  location  was  not 
appropriate  for  training,  due  to  the  many  on-going  distractions. 

The  shipboard  duty  stations  have  more  severe  potential  environment  problems,  such  as: 

•  Noise.  Including  distractions  from  normal  work  noises  (e.g.,  machinery,  repair 
activities),  the  ship’s  public  address  system  announcements,  fire  drills,  and  man 
overboard  drills.  The  impact  of  noise  would  vary  considerably,  depending  on  its 
characteristics  (e.g.,  intensity). 

•  Heat 

•  Vibration 

•  Ship  motion.  This  was  stated  as  a  potential  problem  on  smaller  vessels  during  normal 
seas,  depending  on  the  vessel’s  size.  It  was  generally  not  viewed  as  a  problem  for  larger 
vessels,  even  in  moderate  seas.  Nor  was  it  viewed  as  a  problem  when  tied  up  alongside  a 
pier. 

•  Ship  power  surges.  These,  which  may  occur  often,  could  interrupt  the  training  (e.g., 
lighting  fluctuations). 

5.3.4  Potential  Solutions  to  Training  Environment  Problems 

The  training  environment  problems  identified  previously,  can  be  solved  by  carefiil  selection  of 
the  training  location  space,  and  strict  management  of  training  time. 

5.3.4.1  Training  Space 

A  training  space  should  be  provided  for  each  student  that  eliminates  the  surroundings  problems, 
and  also  reduces  the  potential  for  student  interruptions.  The  availability  of  such  spaces  should  be 
addressed  centrally  within  the  USCG,  to  identify  appropriate  solutions  for  every  type  of  duty 
station  (e.g.,  vessel,  Headquarters).  The  training  space  should  provide  surroundings  conducive  to 
training,  such  as  a  quiet  space  with  appropriate  lighting,  air  conditioning  and  heating,  necessary 
electrical  outlets,  and  with  sufficient  table-top  space.  The  space  should  be  isolated  from 
incidental  interruptions,  such  as  nearby  noise  and  public  address  announcements.  The  ability  to 
meet  these  requirements  will  differ  across  duty  stations,  and  hence  the  particular  solutions  should 
be  left  to  the  local  command’s  discretion.  Several  suggestions  for  appropriate  training  space 
were: 

•  Away  from  the  duty  station.  Training  may  be  conducted  away  from  the  duty  station  to 
eliminate,  or  reduce,  student  interruptions.  The  rationale  suggests  that  if  the  student  is 
not  nearby  the  unit,  the  student  cannot  be  interrupted  or  seized  for  other  activities. 
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•  Available  offices  of  support  unit.  Support  units  generally  have  office  space  available 
for  training.  These  spaces  would  usually  satisfy  the  general  environment  issues,  with  the 
possible  exception  of  interruptions  and  available  time,  which  require  management. 

•  Pierside  office,  for  ships.  This,  of  course,  requires  available  office  space  pierside.  Ship 
personnel  could  use,  for  example,  available  office  and  training  facilities  at  support  units. 
Each  of  the  two  Electronic  Support  Units  (ESU)  interviewed  had  appropriate  space  that 
could  be  used  for  duty  station  training,  including  a  training  laboratory  with  multimedia 
desktop  computers. 

•  Nearby  support  unit,  for  ships.  This  would  be  similar  to  the  pierside  office  addressed, 
but  may  require  some  travel  by  the  student.  If  the  travel  was  substantial  (i.e.,  more  than 
30  minutes),  this  solution  may  affect  the  training  strategy.  Many  support  units,  according 
to  one  person  interviewed,  are  within  a  four-hour  drive  of  duty  stations;  while,  some 
require  up  to  an  eight-hour  drive. 

•  Quiet  space  onboard.  Suggestions  given  for  appropriate  spaces  on  board  differed  for  the 
different  ships.  The  general  opinion  was  that  appropriate  training  spaces  would  be  more 
difficult  to  locate  on  small  ships,  in  comparison  with  larger  ships,  but  they  could  often  be 
arranged.  Suggestions  included: 

•  Radio  room 

•  Officer’s  office  and  stateroom 

•  Messroom 

•  Lounge 

•  Student’s  shop 

•  Other  ship  offices  and  spaces  that  can  be  made  available. 

•  Learning  center  at  each  unit.  The  learning  center  would  require  providing  a  dedicated 
space  at  each  unit,  for  the  sole  purpose  of  training  (Note,  the  space  could  be  shared  with 
other  functions,  which  would  require  some  compromise).  A  dedicated  space  could  more 
easily  furnish  necessary  environment  characteristics,  and  be  amenable  to  firm  control. 
This  suggestion  would  be  most  difficult  on  ships,  especially  the  smaller  ships,  due  to 
space  limitations. 

•  Student’s  home.  This  suggestion  was  viewed  positively  by  all  persons,  as  a  means  of 
greatly  reducing  interruptions  and  achieving  a  good  training  environment.  Training  at 
home,  however,  would  not  be  an  effective  option  for  all  students.  Furthermore,  this 
policy  would  require  portable  laptop  computers,  rather  than  desktop,  for  CBT  courses. 
This  policy  would  also  place  an  administrative  burden  on  the  local  unit,  for  keeping  track 
of  personnel  and  training  materials  (but,  perhaps  no  more  than  some  of  the  other  options). 
Nevertheless,  it  is  a  potentially  good  alternative  for  many  students. 

•  Local  library.  This  would  be  similar  to  the  student  training  at  home,  minus  family 
related  disruptions. 

5.3.4.2  Training  Time  Management 

An  appropriate  training  space  that  provides  a  physical  barrier  to  contact  by  others,  such  as 
training  at  home,  would  certainly  assist  in  solving  the  training  interruption  problem.  However, 
the  problem  of  training  time  management  would  still  remain.  It,  and  the  available  time  problem, 
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requires  personnel  time  management  actions  to  solve.  For  nonresident  CBT  to  succeed,  duty 
station  training  activities  must  be  treated  like  any  other  work  task.  They  must  be  scheduled  in 
advance,  and  conducted  on  a  regular  basis,  integrated  with  the  normal  duty  station  work  tasks. 
Furthermore,  training  tasks,  like  other  work  tasks,  have  certain  associated  conditions  that  must  be 
met,  such  as  no  interruptions.  Management  should  be  responsible  for  creating  and  maintaining 
an  appropriate  training  environment. 

Most  persons  interviewed  stated  that  the  management  of  training  time  at  duty  stations  should  not 
be  a  problem.  But,  they  also  stated  that  appropriate  procedures  and  guidance  must  be  effected  to 
assure  that  an  environment  conducive  to  training  is  established  and  maintained  at  duty  stations. 
Furthermore,  their  suggestions  pertained  to  the  complete  Coast  Guard  command  structure,  from 
Headquarters  to  Districts,  to  duty  stations,  and  finally  to  supervisors  at  the  unit: 

•  Schedule  training  like  any  other  work  task. 

-  Schedule  all  time  blocks  for  a  CBT  course  prior  to  the  start  of  the  course,  like 
scheduling  for  a  long-term  project. 

-  Provide  strict  controls  on  the  scheduling  and  conduct  of  training. 

-  Enforce  the  training  schedule.  Do  not  permit  changes,  except  for  real  emergencies. 

•  Training  Time  Periods.  Appropriate  times  should  be  scheduled  for  training  during  the 
work  day.  Students  should  not  be  expected  to  train  in  their  spare  time,  or  on  their  own 
time.  The  student  must  be  given  the  necessary  amount  of  time  during  normal  work  hours 
to  achieve  the  requisite  training.  This  should  generally  result  in  much  less  student  time 
than  equivalent  instructor-led  resident  training,  providing  a  net  gain  in  available  work 
time  for  the  unit.  Certain  time  periods  may  be  particularly  conducive  to  training  at 
individual  duty  stations.  These  should  be  taken  advantage  of.  For  example,  several 
suggestions  were: 

Any  reasonable  block  of  time  during  the  work  day  (e.g.,  2-hour  block), 

scheduled  in  accordance  with  other  tasks 

Slack  periods  during  extended  deployments 

Slack  periods  during  long  transits 

While  on  standby  duty  in  port 

Several  persons  suggested  that  the  staff  of  a  duty  station  should  receive  some  of  the 
benefit  achieved  by  the  reduction  in  overall  training  time  resulting  from  training  at  the 
duty  station  (e.g.,  if  bottom-line  time  savings  accrue  to  the  unit  as  a  result  of  substantial 
nonresident  CBT,  then  the  unit  staff  should  realize  a  corresponding  workload  reduction). 
This  assumes  that  the  unit  workload  would  not  increase  in  direct  response  to  the 
nonresident  CBT  time  savings. 

•  Do  not  permit  interruptions  of  students  while  training.  Allow  interruption  of  students 
only  for  real  emergencies. 

•  Establish  firm  policies  for  conduct  of  duty  station  training.  The  policies  should: 

-  Apply  at  all  levels  within  the  USCG,  including  Headquarters  directorates.  Districts 
and  duty  stations. 

Create  an  environment  at  duty  stations  conducive  to  training. 
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-  Prevent  individuals  or  commands  from  circumventing  the  required  duty  station 
training  policies.  That  is,  the  policies  should  define  the  requirements  for  conduct  of 
training  at  duty  stations,  and  should  insulate  the  duty  station  from  interference  in 
meeting  those  requirements. 

-  Provide  flexibility  to  the  duty  station  for  configuring  and  controlling  their  training 
process,  in  accordance  with  their  needs.  That  is,  the  established  policies  should 
specify  requirements  to  assure  effective  training,  but  also  allow  for  local  flexibility  in 
meeting  the  requirements.  Local  unit  flexibility  in  achieving  their  training  needs  is  a 
major  factor  in  support  of  nonresident  CBT. 

•  Top  down  support  for  training  policies.  Firm  support  for  nonresident  CBT  must  be 
given,  and  demonstrated,  across  the  USCG.  This  support  should  be  interjected  into 
USCG  management  preparation  activities  (e.g.,  Prospective  Commanding  Officer 
Course),  and  actively  marketed.  It  should  be  reflected  in  command  directives  at  all 
levels. 

5.4  STUDENT  MOTIVATION 

It  is  estimated,  based  on  the  interviews,  that  approximately  half  the  students  will  prefer 
nonresident  CBT  to  traditional  instructor-led  resident  training.  Reasons  for  preferring 
nonresident  CBT  were  given  earlier  (Section  5.1,  Overall  Opinion  of  Nonresident  Training). 
Reasons  for  preferring  instructor-led  resident  training  varied,  some  of  which  have  been 
mentioned  throughout  this  section.  Generally,  resident  training  would  be  preferred  as  a  break 
from  the  daily  work  routine,  and  for  the  opportunity  to  travel.  Another  important  reason  cited 
was  the  training  methodology,  which  included  the  individual’s  learning  style  and  dislike  for 
computers  (e.g.,  dislike  or  unfamiliarity).  Regardless  of  the  reasons,  a  student  motivation 
problem  should  be  expected  to  accompany  nonresident  CBT,  potentially  affecting  many  students 
to  varying  degrees.  Included  in  this  problem  should  be  those  students  who  may  prefer 
nonresident  CBT,  but  who  would  have  a  motivation  problem  for  other  reasons. 

The  challenge  facing  the  Coast  Guard  training  community  is  how  to  motivate  all  students  to  train 
effectively  and  efficiently  when  in  a  self-paced  nonresident  course.  An  instructor,  who  normally 
deals  with  this  problem,  would  not  be  present.  The  several  suggestions  were: 

•  Training  facilitator.  Have  a  duty  station  facilitator  oversee  and  interact  with  each 
student.  The  facilitator  would  be  tasked  with  motivating  the  student.  The  facilitator  is 
addressed  in  more  detail  later  in  this  section  (Section  5.7.3,  Facilitator). 

•  Supervisor  responsibility.  The  student’s  immediate  supervisor  would  be  responsible  to 
motivate  the  student  to  perform  well  in  the  course,  similar  to  the  supervisor’s 
responsibility  in  overseeing  the  student’s  work.  Some  persons  suggested  that  student 
motivation  be  left  to  the  unit,  which  is  similar  to  placing  that  responsibility  with  the 
student’s  immediate  supervisor. 

•  Provide  incentives  to  the  student.  Incentives  were  suggested  for  student  performance, 
including  proficiency  (learning  and  test  performance)  and  efficiency  (course  completion 
time),  as  follows: 
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-  Time  off.  If  the  student  finishes  the  course  in  less  time  than  planned,  with  acceptable 
proficiency,  the  student  would  be  given  time  off  in  proportion  to  the  training  time 
saved.  The  student  could  be  given  all  of  the  time  saved,  or  a  portion  of  the  time  saved 
(e.g.,  50%,  on  the  basis  of  50/50  sharing  with  the  unit).  If  shared,  the  time  savings 
could  have  motivational  benefits  across  the  unit,  since  other  staff  would  benefit  by 
having  the  student  available  for  more  work  as  a  result  of  early  course  completion. 
Course  grade.  The  awarding  of  a  course  grade  was  suggested  as  having  real 
incentive  value  in  motivating  students  to  perform  well. 

-  Notations  of  course  grade  and  completion  time,  to  be  placed  in  the  student’s 
personnel  file  and  on  a  course  completion  certificate. 

-  Proficiency  pay.  This  would  be  a  financial  incentive  for  the  course  grade  and  the 
course  completion  time.  It  could  be  a  one-time  bonus  paid  at  the  completion  of  the 
course,  similar  to  the  tuition  reimbursement  policy  of  some  companies  (i.e.,  100% 
reimbursement  for  an  A,  75%  for  a  B,  50%  for  a  C,  and  0%  for  a  D  or  F). 

5.5  TRAINING  PROCESS 

Potential  problems  were  identified,  and  suggestions  provided,  for  several  topics  relating  to  the 
training  process.  The  suggestions  provide  guidance  for  the  conduct  of  training,  which  is  an 
essential  part  of  a  nonresident  implementation  plan. 

5.5.1  Training  Session 

5.5.1. 1  Daily  Training  Session 

A  typical  CBT  course  would  provide  hours  of  training  to  a  student,  achieved  during  multiple 
training  sessions  spread  over  a  series  of  days.  The  training  sessions  would  be  interspersed  with 
work  at  the  unit,  during  the  typical  workday.  This  approach  would  achieve  distributed  training, 
which  is  preferable  to  condensed  training  (i.e.,  in  which  the  student  engages  only  in  training 
activities  until  the  CBT  course  is  completed).  Suggestions  for  the  training  sessions  are: 

•  Maximum  training  session  length  of  four  hours;  two  hours  preferred.  If  the  student 
trained  for  1  to  2  hours  a  day,  substantial  time  would  be  available  for  daily  work  tasks. 
The  training  session  length  (maximum  and  minimum)  should  be  keyed  to  the  training 
material,  such  that  the  student  would  stop  at  a  logical  stopping  point,  rather  than  simply  a 
time  point.  If  the  training  session  is  expected  to  extend  for  several  hours,  then  breaks 
should  be  provided  approximately  every  hour.  The  breaks  should  coincide  with  logical 
pause-points  in  the  course  (e.g.,  after  completion  of  a  problem). 

•  Minimum  training  session  length  of  1  hour.  (See  previous  discussion) 

•  Scheduling  of  training  periods  left  to  discretion  of  duty  station.  The  morning  hours 
were  generally  preferred  for  training,  although  any  time  of  day  is  acceptable.  Scheduling 
of  training  was  suggested  on  a  case-by-case  basis,  as  well  as  simultaneously  for  the  whole 
unit  or  groups  of  staff  (e.g.,  a  block  of  time  when  most  staff,  or  a  group,  would  be 
engaged  in  training). 
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•  Multiple  training  sessions  could  be  conducted  each  day  for  a  student,  such  as  early 
morning  and  mid  afternoon.  They  should  be  separated  by  substantial  time  (e.g.,  several 
hours),  and  have  a  short-to-medium  length  (e.g.,  no  more  than  2  hours). 

5.5.1.2  Course  Length 

The  length  of  a  CBT  course  will  depend  on  the  training  objectives,  training  strategy,  and  other 
factors.  Resident  courses  of  varying  lengths  may  be  converted  to  a  nonresident  CBT  form,  in 
whole  or  part.  Some  of  these  may  be  quite  long,  currently  requiring  several  weeks  of  training. 
Since  the  training  is  expected  to  be  conducted  in  a  distributed  manner  at  the  duty  station,  as 
discussed  previously,  long  courses  may  be  difficult  to  administer.  For  example,  a  20-hour  CBT 
course  may  require  two  weeks  to  administer  (two-hour  sessions,  five  days  a  week).  A  40-hour 
course  similarly  conducted  might  require  about  a  month  to  complete. 

A  range  of  opinions  was  given  for  the  maximum  length  of  a  CBT  course  to  be  conducted  at  duty 
stations,  from  16  to  40  hours.  Since  a  major  advantage  of  CBT  is  its  allowance  of  self-paced 
training,  a  nominal  40-hour  course  may  take  some  students  much  longer  to  complete;  or,  it  may 
be  completed  in  less  time.  Shorter  completion  times  would  not  present  a  problem  for  the  duty 
station.  The  longer  completion  times,  however,  would  pose  an  administrative  problem. 
Nonresident  CBT  course  lengths  should  be  limited  to  a  reasonable  average  time.  A  course 
should  not  exceed  40-hours  in  length.  And,  a  more  reasonable  maximum  course  length  would  be 
around  25  hours. 

Course  conversions  (i.e.,  converting  existing  resident  courses  to  nonresident  CBT  form)  should 
aim  for  a  reasonable  CBT  course  length.  This  may  necessitate  dividing  the  original  course  into 
several  shorter  courses  when  converting  to  the  nonresident  CBT  form.  For  example,  a  current 
six-week  course  may  have  to  be  converted  into  three  or  more  CBT  courses.  This  conversion 
issue  may,  therefore,  affect  the  structure  of  course  offerings  in  the  USCG  (e.g.,  one  course 
converted  into  several  shorter  courses;  course  prerequisite  issues;  qualification  and  rating  issues). 

5.5.2  Student-Computer  Interface 

Although  many  USCG  personnel  are  familiar  with  using  computers,  the  level  of  computer 
proficiency  is  likely  to  vary  widely.  Furthermore,  some  persons  will  not  be  comfortable  in 
training  with  computer  media,  as  noted  earlier  (See  Section  5.4,  Student  Motivation).  Several  of 
the  implementation  suggestions  in  other  sections  partially  address  this  problem.  Two  additional 
suggestions  were  provided: 

•  Develop  and  specify  a  common  student-computer  interface  for  all  USCG  CBT 
courses.  This  interface  should  not  be  limiting  to  developers,  but  should  provide 
fundamental  standards  that  would  result  in  a  similar  interface  across  courses.  Its  goal 
should  be  to  achieve  rapid  student  familiarity  (e.g.,  recognition  of  response  procedures, 
features  and  controls)  when  starting  a  new  course.  Since  students  would,  presumably, 
take  many  nonresident  CBT  courses  over  their  career,  having  the  basic  knowledge  and 
skill  set  to  immediately  interact  in  an  effective  manner  with  a  new  course  can  have 
distinct  efficiency,  effectiveness  and  motivational  benefits.  The  interface  standard  should 
not  inhibit  interface  improvement.  It  should  be  linked  with  current  commercial  person- 
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computer  interface  standards,  CBT  course  student  interaction  approaches  (e.g.,  selection 
and  operation  of  tools  using  a  mouse;  multiple  choice  question  presentation;  location  of 
HELP  function;  window  manipulation),  and  allowed  to  evolve  along  with  the  computer¬ 
training  industry. 

•  Develop  a  student  training  guide.  This  guide  would  be  specific  to  nonresident  (e.g., 
CBT,  IVT)  training  at  duty  stations.  It  would  apply  across  the  range  of  nonresident 
courses.  It  would  provide  both  general  training  guidance,  and  serve  as  a  handbook  for  the 
student  interface  with  various  training  media,  particularly  the  student-computer  interface. 
This  type  of  a  guide  is  particularly  important  due  to  the  level  of  responsibility  the  student 
bears  for  nonresident  training  (e.g.,  self  paced;  may  be  located  remotely  from  the  duty 
station). 


5.5.3  Student  Help 

A  drawback  of  nonresident  training  is  the  absence  of  an  instructor  to  help  the  individual  student 
when  particular  learning  difficulties  are  encountered.  Sources  of  technical  assistance  must  be 
available  to  students  for  each  nonresident  course  (i.e.,  this  is  in  addition  to  general  assistance, 
which  would  be  provided  by  their  unit;  see  Section  5.7.3,  Facilitator).  The  following  two 
suggestions  were  postulated  as  able  to  satisfy  this  problem. 

•  Assistance  from  nearby  senior  persons.  Have  nearby  senior  persons  available  to 
answer  questions  and  provide  clarification  to  students.  Senior  persons  in  the  student’s 
unit,  if  available,  would  be  most  appropriate.  Many  persons  felt  that  senior  persons  on 
other  nearby  units  could  also  provide  this  type  of  assistance,  either  by  having  the  student 
go  to  the  other  unit,  or  by  telephone.  This  type  of  assistance  is  familiar  to  most  USCG 
units,  in  the  form  of  on-the-job-training.  This  suggestion  would  expand  that  concept.  It 
would  require  receptivity  on  the  part  of  the  senior  persons,  and  their  units.  As  with  all  of 
the  suggestions  in  this  report,  to  be  effective,  this  type  of  assistance  must  be  addressed  in 
formal  USCG  policy. 

•  Hotline  to  a  Subject  Matter  Expert  (SME).  All  persons  felt  that  technical  help  could 
be  provided  to  individual  students  through  a  telephone  hotline.  A  hotline  would  be 
provided  for  each  nonresident  course.  Ideally,  the  hotlines  would  be  accessible  24  hours 
a  day,  although  this  may  not  be  feasible.  Internet  and  email  access  was  suggested  as 
alternatives.  A  combination  of  voice  telephone,  (e.g.,  certain  hours  of  the  day),  email, 
and  internet  may  provide  a  good  mix  of  help.  Note,  cost  and  accessibility  of 
communication  links  may  be  significant  issues  for  some  duty  stations  (e.g.,  vessels  at 
sea).  The  MARSAT  link  was  cited  as  expensive,  and  having  a  daily  minutes-per-day 
access  limit.  Several  persons  also  suggested  integrating  the  training  help  hotline  with  an 
equipment  support  hotline. 

5.5.4  Hands-On  Training 

Hands-on  training  may  be  an  important  part  of  certain  nonresident  courses.  The  necessity  for  a 
hands-on  training  course  component,  as  well  as  its  content,  will  depend  on  the  particular  course. 
It  was  considered  important  for  the  AN/WSC-3  CBT  course,  with  a  two-hour  hands-on  training 
module  provided  to  all  students.  Its  purpose  in  the  AN/WSC-3  CBT  course  was  to  assist  the 
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student  in  transitioning  from  the  CBT  medium  to  the  actual  hardware  (e.g.,  familiarize  the 
students  with  the  AN/WSC-3  hardware;  locate  test  points). 

Most  persons  interviewed  felt  that  hands-on  training  is  necessary,  and  could  be  effectively 
conducted  in  the  field,  at  an  operational  unit.  Some  suggested  that  this  type  of  training  at  the 
field  unit  would  have  the  added  benefit  of  using  equipment  configured  for  operational  use  (i.e., 
rather  than  a  generic  configuration  in  a  training  lab).  The  hands-on  training  should  be  conducted 
in  the  presence  of  a  senior  staff  member  of  the  unit  where  the  hardware  is  located,  who  would 
oversee  the  student’s  activities.  The  general  opinion  was  that  the  senior  individual  need  not  be 
qualified  in  the  student’s  area  of  training,  although  it  would  be  advantageous. 

All  persons  interviewed  felt  that  hands-on  training  would  be  important  for  some  courses,  but  not 
necessarily  for  all  courses.  Several  issues  addressed  were: 

In  what  courses  is  hands-on  training  necessary? 

How  much  hands-on  training  is  necessary? 

What  should  be  the  content  of  the  hands-on  training? 

Where,  when  and  how  should  the  hands-on  training  be  conducted? 

Courses  Requiring  Hands-On  Training.  Hands-on  training  was  generally  viewed  as  important 
for  courses  involving  the  training  of  manual  skills  (e.g.,  maintenance  and  operation  courses),  and 
not  for  knowledge-only  courses  (e.g.,  administration  courses).  Exercising  was  cited  as  important 
in  this  regard.  It  should  be  noted  that  findings  from  the  student  and  facilitator  questionnaires 
identified  the  need  for  more  practice  problems  in  the  AN/WSC-3  CBT  course,  which  equates  to 
more  exercising.  Hence,  part  of  the  identified  need  for  hands-on  training  may  be  satisfied  by 
more  exercising  in  the  CBT  media. 

A  nonresident  training  implementation  plan  should  address  the  issue  of  which  courses  require  a 
hands-on  training  component.  The  ensuing  issues  of  the  particular  hands-on  training  course 
content,  and  the  amount  of  such  training,  would  also  need  to  be  determined  for  each  course. 

Amount  of  Hands-On  Training.  Although  it  was  difficult  for  the  interviewed  persons  to 
estimate  the  amount  of  hands-on  training  that  would  be  required,  due  to  the  many  different 
courses  that  may  be  possible,  the  2-hours  provided  in  the  AN/WSC-3  CBT  course  was  generally 
cited  as  a  good  role  model.  This  means  that  the  hands-on  training  may  be  brief  for  a  maintenance 
and  operations  course. 

Location  and  Timing  of  Hands-On  Training.  The  location  in  which  the  hands-on  training 
would  be  provided  is  a  particularly  difficult  issue,  and  affects  the  timing  and  content  of  such 
training.  If  the  appropriate  hardware  is  available  at  the  student’s  duty  station,  most  persons  felt 
that  this  would  be  the  best  place  to  conduct  hands-on  training.  After  completing  the  CBT  part  of 
the  course,  the  student  would  then  go  through  the  hands-on  part  at  the  duty  station  during  the 
next  training  session. 

The  difficulty  arises  when  the  required  equipment  is  not  located  at  the  student’s  duty  station,  for 
example,  if  the  student  is  being  trained  prior  to  transferring  to  the  next  duty  station.  The 
suggestions  to  address  this  problem  were: 
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•  Delay  the  hands-on  training  part  until  the  student  reports  to  the  next  duty  station, 

which  would  have  the  necessary  equipment.  This  was  the  solution  favored  by  most 
persons.  It  is  viable  if  the  student  will  transfer  shortly  after  completion  of  the  CBT  part 
of  the  course.  If,  on  the  other  hand,  months  pass  before  the  student  transfers,  retention  of 
knowledge  and  skills  may  become  a  problem.  Note,  retention  would  usually  be  a 
problem  when  there  is  a  delay  between  course  completion  and  start  of  related  work, 
whether  or  not  hands-on  training  was  accomplished.  Hence,  the  delay  may  not  have  any 
greater  adverse  impact  than  that  currently  being  experienced.  Several  persons  noted  that 
a  student  may  have  a  difficult  time  taking  an  important  CBT  course  shortly  after  arriving 
at  a  new  duty  station,  because  of  the  student’s  unfamiliarity  with  the  unit,  personal  issues 
associated  with  the  recent  move,  and  other  learning  demands  that  may  be  placed  on  new 
personnel  (e.g.,  damage  control  training). 

•  Delay  the  CBT  course  until  the  student  reports  to  the  new  duty  station.  The  course 
would  then  be  taken  at  a  duty  station  that  has  the  necessary  equipment  for  hands-on 
training.  An  indirect  benefit  of  this  alternative  is  that  the  student  would  be  trained  much 
nearer  the  time  the  knowledge  and  skills  would  be  used.  This  alternative  would 
necessitate  re-structuring  the  relationship  between  training  and  transferring,  as  regards 
pipeline  training.  The  person  would  report  to  the  new  duty  station  before  receiving  the 
requisite  training.  The  transferring  person  may  experience  difficulty  taking  the  course  at 
the  new  duty  station,  as  noted  in  the  preceding  suggestion. 

•  Conduct  the  CBT  course  at  a  shore  unit  nearby  the  student’s  new  duty  station.  The 
student  would  transfer  out  of  the  current  duty  station  to  the  unit  nearby  the  new  duty 
station,  to  participate  in  nonresident  CBT.  The  nonresident  CBT  would  be  conducted  at 
the  shore  unit  (e.g.,  Engineering  Support  Unit).  After  completion  of  the  training,  the 
student  would  report  to  the  new  duty  station,  at  which  time  the  hands-on  training  would 
be  conducted.  If  the  appropriate  equipment  is  available  at  the  temporary  shore  unit,  the 
hands-on  training  could  be  conducted  at  that  location. 

•  Shoreside  support  units  could  be  equipped  to  provide  the  hands-on  training  (As  well 
as  provide  space  for  students  to  conduct  the  CBT  course,  as  discussed  above). 

Appropriate  equipment  would  be  necessary  at  the  unit  to  conduct  the  hands-on  training. 

•  Have  the  hands-on  training  accomplished  at  a  unit  nearby  the  student’s  duty 
station.  Many  persons,  although  not  all,  felt  that  this  alternative  would  work.  Those  that 
did  not  believe  this  to  be  a  viable  alternative  suggested  that  the  unit  with  the  equipment 
may  be  reluctant  to  allow  an  unknown  person  to  work  on  their  equipment.  This  attitude 
would  likely  exist  in  at  least  some  situations,  relegating  this  alternative  to  less  feasible. 

•  Let  the  student’s  unit  decide  on  how  the  required  hands-on  training  will  be  achieved. 
This  alternative  was  suggested  by  a  number  of  persons,  noting  that  several  of  the  above 
alternatives  are  viable,  and  that  the  best  will  depend  on  the  situation  at  the  specific  unit. 

The  bottom-line  is  that  hands-on  training  is  expected  to  be  necessary  for  certain  courses.  The 
amount  of  hands-on  training  that  is  required  should  be  relatively  small  for  most  courses  (e.g.,  1 
or  2  hours  for  a  course  like  the  ANAVSC-3  CBT  course),  although  not  all  courses.  The  required 
hands-on  training  can  be  achieved  at  a  USCG  unit.  Several  viable  approaches  are  available  for 
achieving  the  hands-on  training.  The  approach  used  may  have  to  be  determined  by  the 
circumstances  at  the  particular  duty  station. 
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Content  of  Hands-On  Training.  It  was  suggested  that  most  of  the  training  content  in  a  course 
could  be  conducted  in  the  CBT  media,  with  the  hands-on  training  focused  on  transition  and 
validation  of  skills  and  knowledge.  Suggestions  for  hands-on  training  content,  in  the  context  of 
maintenance  training,  were: 

Preventive  Maintenance  Service  (PMS) 

Testing  of  boards 

Alignments 

Front  panel  controls 

Location  of  test  points  and  adjustment  points 

The  conduct  of  hands-on  training  in  the  field  raises  the  issue  of  the  extent  to  which  a  student 
would  be  allowed  to  interact  with  operational  equipment.  Most  persons  felt  that  having  the 
student  operate  and  perform  basic  maintenance  procedures  (e.g.,  alignments)  on  operational 
equipment  would  be  acceptable,  especially  since  the  student  would  have  essentially  completed 
the  course  prior  to  the  hands-on  part. 

Different  opinions  were  expressed  on  the  viability  of  inserting  faults  into  operational  equipment 
for  training  or  testing.  Many,  although  not  all,  persons  suggested  that  operational  units  would 
not  allow  faults  to  be  inserted  into  their  operating  equipment.  Several  persons,  on  the  other  hand, 
suggested  that  insertion  of  faults  would  be  viable  at  many  units,  especially  if  the  fault  insertion 
was  accomplished  by  installing  a  faulty  component  (e.g.,  rigged  board)  that  would  not  affect 
other  components;  and  if  the  unit  had  multiple  sets  of  the  equipment  (e.g.,  large  cutters  often 
have  multiple  AN/WSC-3  UHF  transceivers). 

Regardless  of  the  hands-on  training  content,  formal  procedures  are  deemed  necessary  for  the 
hands-on  training  associated  with  any  course.  The  procedures  would  provide  specific  guidance 
to  the  student,  as  well  as  the  senior  person  overseeing  the  hands-on  training. 

5.5.5  Student  Input  Characteristics 

Instructors  in  resident  courses  often  adapt  at  least  part  of  their  instruction  to  the  characteristics  of 
the  students.  Thus,  although  courses  may  have  specified  student  input  characteristics  (i.e., 
prerequisite  skills  and  knowledge)  the  range  of  skills  and  knowledge  can  vary  widely  across  a 
class.  CBT  course  adaptability  to  the  student’s  entering  proficiency,  would  be  similar.  However, 
high  degrees  of  adaptability  in  CBT  courses  are  expensive  to  develop.  Consequently,  most  CBT 
courses  are  somewhat  rigid,  being  developed  for  a  specific  level  of  student  input  characteristics. 
This  is  an  important  issue  to  consider  in  developing  the  nonresident  structure  of  courses.  Two 
alternative  solutions  were  suggested. 

•  Automatic  Diagnostic  Evaluation  and  Branching.  Build  adaptability  into  the  CBT 
course.  This  would  require  diagnostic  evaluation  of  the  student’s  relevant  input 
characteristics  at  the  start  of  the  course,  with  automatic  tailoring  of  the  subsequent 
training  to  fit  the  student’s  profile.  For  example,  the  course  might  automatically 
determine  which,  if  any,  preliminary  training  modules  should  be  conducted  prior  to  the 
main  course  content,  based  on  the  diagnostic  evaluation.  It  might  also  order  the 

61 


presentation  sequence  of  these  preliminary  training  modules,  as  well  as  those  of  the  main 
course.  Finally,  the  course  may  automatically  select  or  change  the  sequence  of  training 
modules  at  any  time  during  the  training,  in  correspondence  with  student  performance. 
Diagnostic  evaluation  and  branching  of  this  type  would  allow  a  wider  range  of  student 
input  characteristics,  but  is  more  expensive  to  develop  than  a  course  based  on  one  serial 
path  through  the  modules. 

•  Stringent  Student  Prerequisites.  The  required  student  prerequisites  for  a  nonresident 
course  that  is  not  adaptable  should  be  stringent.  Otherwise,  student  time  may  be  wasted 
by  engaging  in  a  course  for  which  the  student  does  not  possess  the  minimum  skills  and 
knowledge. 

5.5.6  Testing 

Testing  is  concerned  with  two  issues:  1)  testing  as  part  of  the  training  course,  to  assess  student 
progress,  and  strengths  and  weaknesses;  and  2)  testing  of  the  student  pursuant  to  certification  that 
the  student  has  passed  the  course  and  is  capable  of  proficient  performance.  Certification,  and  its 
associated  testing,  is  addressed  in  the  next  section.  Testing  in  support  of  the  training  process  is 
addressed  here. 

Testing  was  an  issue  because  much  of  the  testing  in  resident-based  operation  and  maintenance 
courses  is  accomplished  with  actual  equipment  in  a  lab.  It  was  generally  felt  that  testing  in  the 
computer-display  medium,  similar  to  the  training  and  problem  solving  activities  during  the 
course,  would  be  satisfactory. 

If  hands-on  testing  were  necessary,  the  suggestions  were  for  it  to  be  integrated  with  the  hands-on 
training.  The  problems  associated  with  such  hands-on  testing  would  be  similar  to  those 
discussed  previously  regarding  hands-on  training. 

5.6  STUDENT  CERTIFICATION 


This  is  an  important  and  diverse  issue  that  must  be  carefully  addressed  in  a  nonresident  training 
implementation  plan,  to  assure  that  students  meet  acceptable  proficiency  standards.  This  issue 
also  overlaps  with  the  testing  and  hands-on  training  issues  previously  discussed. 

5.6.1  Certification  Authority 

It  was  generally  suggested  that  a  central  USCG  authority  (e.g.,  USCG  Institute)  issue  the 
certification  for  each  student,  rather  than  the  duty  station.  The  certification,  including  the 
qualification  code,  would  be  issued  after  the  student  has  completed  the  course  and  all  requisite 
certification  testing. 

Additional  suggestions  for  the  certification  authority  included:  1)  student’s  duty  station 
commanding  officer;  2)  duty  station’s  education  officer  or  training  officer;  and  3)  the  course’s 
Subject  Matter  Expert  (SME)  (i.e.,  who  would  be  available  through  the  student’s  help  hotline). 
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5.6.2  Evaluation  of  Student  Proficiency 


Issues  addressed  with  regard  to  student  evaluation  for  certification  were:  1)  Need  for  student 
certification  testing;  2)  Certification  test  media;  and  3)  Location  and  administration  of  the 
certification  testing. 

Need  for  Student  Certification  Testing.  Many  of  the  duty  station  persons  interviewed  felt  that 
some  form  of  certification  testing  should  be  required,  after  the  student  completed  the  course. 
Usually  this  opinion  was  associated  with  the  need  to  demonstrate  proficiency  on  the  actual 
hardware  (i.e.,  hands-on  testing).  It  was  also  suggested  that  this  type  of  testing  would  be 
necessary  at  least  during  the  early  stages  of  widespread  nonresident  training,  to  demonstrate 
student  proficiency  to  the  units.  Certification  testing  of  each  student  could  later  be  replaced  with 
certification  of  the  course  (see  next  paragraph). 

Some  persons  suggested  that  the  student  should  be  certified  as  a  result  of  completing  the  course, 
and  should  not  require  a  separate  certification  test.  This  approach  would  require  certification  of 
the  course,  rather  than  the  students.  All  students  who  successfully  complete  the  course  would 
have  passed  the  tests  associated  with  the  course,  and  therefore  would  have  already  demonstrated 
their  proficiency.  In  essence,  the  certification  test  would  be  built  into  the  course. 

Certification  Test  Media.  The  typical  certification  test  would  have  the  student  perform  tasks 
similar  to  those  required  in  the  operational  environment  (e.g.,  troubleshooting  the  AN/WSC-3). 
Two  basic  types  of  test  media  were  suggested:  1)  computer  media  used  during  the  course;  and  2) 
actual  equipment.  Although  more  persons  suggested  testing  on  the  actual  equipment,  a  number 
of  persons  suggested  that  testing  in  the  computer-based  training  media  would  be  satisfactory. 

It  is  recommended  that  the  USCG  consider  development  and  empirical  evaluation  of  computer- 
based  tests  for  the  initial  courses  converted  to  a  nonresident  form.  This  approach  would  serve  to 
develop,  evaluate  and  demonstrate  the  viability  of  computer-based  testing  in  lieu  of  testing  on  the 
actual  equipment  (i.e.,  assuming  that  such  testing  would  be  valid  and  reliable).  The  initial 
students  could  be  tested  with  both  media,  providing  the  basis  for  validity  and  reliability 
assessments. 

Location  a«H  Administration  of  Certification  Test.  If  the  certification  test  is  to  be  conducted 
using  the  computer-based  media,  it  could  take  place  in  the  training  location.  If,  on  the  other 
hand,  the  certification  test  is  to  be  conducted  with  actual  hardware,  the  location  issues  discussed 
above  regarding  hands-on  training  apply  (as  do  the  alternative  solutions).  Note,  the  person(s) 
involved  with  administration  of  the  certification  test  would  not  necessarily  award  the  student 
certification,  although  they  could  (see  the  above  discussion  about  certification  authority). 

A  variety  of  suggestions  were  made  for  how  the  certification  testing  should  be  administered: 

•  Remote  link  with  an  SME.  This  approach  received  support  by  all  of  the  persons 
interviewed.  The  test  could  use  either  the  computer-based  or  actual  equipment  media, 
with  the  student  performing  activities  directed  by  the  SME.  The  student  would  be 
connected  by  telephone  or  internet  to  the  remotely-located  SME,  who  would  direct  the 
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test,  evaluate  the  results,  and  pass  or  fail  the  student.  A  senior  person  (e.g.,  training 
officer)  would  be  with  the  student  during  all  parts  of  the  test,  to  oversee  the  student’s 
activities  and  provide  a  local  control  person  for  the  SME.  The  test  would  be  performed  in 
an  appropriate  office  or  space,  depending  on  the  test  media  (i.e.,  actual  hardware  or 
computer-based)  and  other  factors  addressed  earlier. 

•  Support  unit  (or  detachment)  staff.  An  appropriately  qualified  individual  at  a  nearby 
support  unit  could  administer  the  test  and  certify  the  student.  This  could  be  accomplished 
at  the  student’s  duty  station,  or  the  support  unit  location. 

•  Student  supervisor  or  other  senior  qualified  person  at  the  duty  station.  Depending 
on  the  test,  the  administrator  may  or  may  not  have  to  be  qualified  in  the  subject  matter. 

•  Other  suggestions  were: 

Commanding  officer  of  the  duty  station 

Education  officer,  or  training  officer  at  the  duty  station 

-  Qualified  senior  person  from  a  nearby  unit 

-  Contractor  support  team  working  in  the  area.  If  they  are  not  permanently  assigned  to 
the  area,  then  testing  would  be  scheduled  with  their  visit  to  the  area. 

-  USCG  support  team  working  in  the  area. 

Whoever  the  test  administrator  is,  that  person  should  be  qualified  to  administer  the  test  (i.e.,  have 
special  training  and  preparation,  the  extent  of  which  will  depend  on  the  test  requirements). 

The  above  suggestions  that  would  be  under  control  of  the  duty  station  would  provide  them  the 
greatest  flexibility  for  training,  testing  and  certifying  their  personnel.  Other  solutions  would 
require  careful  scheduling  of  the  testing,  with  less  flexibility  for  the  duty  station. 

5.7  NONRESIDENT  TRAINING  MANAGEMENT 


Training  management  includes  issues  of  central  versus  local  control  (e.g.,  storage  and 
distribution  of  training  materials),  and  the  local  daily  management  of  training  activities  at  the 
duty  station. 


5.7.1  Duty  Station  and  Central  Control  Issues 

Field  units,  today,  have  little  control  over  when  a  course  is  taught,  and  must  adapt  their  activities 
to  the  course  schedule  if  they  want  a  student  to  receive  the  training.  With  nonresident  training, 
since  the  course  would  be  taught  at  the  unit  (or  at  least  nearby),  and  since  the  training  materials 
could  be  made  available  to  every  unit,  the  duty  station  could  control  the  course  schedule.  Along 
with  this  increased  flexibility  at  duty  stations  resulting  from  nonresident  training,  would  be 
increased  tasking  and  responsibility  to  administer  the  associated  training  functions. 

A  perceived  major  strength  of  nonresident  training,  as  noted  earlier,  would  be  the  ability  of  the 
unit  to  conduct  particular  courses  when  convenient,  and  to  have  their  personnel  trained  in  the 
skills  they  deem  necessary.  The  unit  would  not  be  affected  by  limited  course  quotas,  or  an 
incompatible  course  schedule.  Furthermore,  they  could  achieve  a  greater  degree  of  cross  training 
to  assist  with  anticipated  workload  demands.  Consequently,  persons  interviewed  strongly  urged 
that  these  aspects  of  nonresident  training  be  placed  under  the  control  of  the  duty  station,  namely, 
who  to  train,  what  to  train,  and  when  to  train. 
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Most  persons  recognized  a  need  for  some  degree  of  central  control.  This  was  suggested  to  take 
the  form  of  guidance  on  how  to  conduct  training  at  the  duty  station,  including  general  guidance 
and  course-specific  guidance.  The  differences  between  duty  stations  were  suggested  to 
necessitate  differing  levels  of  central  and  local  control,  with  a  general  conclusion  that  a  mix  of 
control  would  be  desirable,  which  could  be  determined  by  each  duty  station.  The  issues  and 
suggestions  in  this  regard  pertain  to: 

•  Management  of  training  course  materials 

•  Training  and  testing  control 

•  USCG  Institute 

Management  of  Training  Course  Materials.  This  includes  storage  and  distribution  of  training 
media  (including  hardware),  and  configuration  management.  Mixed  opinions  were  received  for 
the  following: 

•  Central  USCG  storage  and  distribution  of  all  training  media.  When  a  duty  station 
wants  a  person  trained,  the  course  would  be  ordered  from  the  central  repository,  and 
immediately  shipped  to  the  unit.  This  turn  around  time  should  be  brief,  perhaps  a  couple 
of  days.  The  central  repository  would,  thus,  be  saddled  with  the  material  management 
function.  At  one  extreme,  the  central  repository  would  supply  all  materials,  including 
hardware,  software,  and  other  materials  (e.g.,  technical  manuals).  At  the  other  extreme, 
the  central  repository  would  manage  only  the  software  and  specific  training  manuals,  but 
not  operation  or  technical  manuals.  A  small  cutter  may  prefer  this  alternative,  with  a 
small  local  administrative  and  storage  burden. 

•  Complete  library  of  USCG  CBT  courses  at  each  duty  station.  Generally,  this  would 
consist  of  a  library  of  compact  disks  (CD’s).  When  a  duty  station  wants  a  person  trained, 
the  training  supervisor  would  get  the  course  CD  from  the  library  and  initiate  the  training. 
This  approach  necessitates  that  all  training  materials  to  be  on  the  course  CD.  Hardware 
and  other  operational  materials  would  also  have  to  be  resident  at  the  duty  station,  or 
obtained  separately.  The  duty  station  would  be  tasked  with  configuration  control  of  its 
course  CD  inventory  (e.g.,  Make  sure  its  courses  are  the  latest  versions).  A  large  cutter 
that  often  goes  on  extended  deployments  would  likely  prefer  this  alternative,  since 
unanticipated  training  needs  could  be  met  while  deployed. 

•  Mixed  approach,  in  which  the  central  repository  functions  as  suggested  above  for  course 
CD’s,  and  in  which  each  duty  station  can  maintain  a  library  of  course  CD’s  as  desired. 
The  duty  station  would  keep  CD’s  for  their  high-use  courses,  and  request  others  from  the 
central  repository.  Navy  vessels,  at  least  some,  use  this  approach.  A  copy  of  relevant 
courses  is  kept  in  the  onboard  library,  for  which  the  training  officer  is  responsible. 

Each  of  the  above  alternatives  has  administrative  burdens  for  the  duty  station.  The  central 
repository  would  likely  have  the  smallest  burden,  but  would  also  be  the  least  flexible  for  the  duty 
station.  The  preferences  for  the  first  two  alternatives  were  split  between  persons  interviewed. 

On  the  other  hand,  most  persons  felt  the  mixed  alternative  would  be  satisfactory. 
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Training  and  Testing  Control.  Several  issues  were  addressed  regarding  the  types  of  central 
controls  that  should  be  applied  to  the  duty  station  training  process,  with  the  following 
suggestions: 

•  Course  offerings.  Scheduling  of  courses  should  be  at  the  discretion  of  the  duty  station. 

•  Training  process.  Guidance  should  be  provided  to  duty  stations,  both  general  for  all 
courses,  and  specific  with  each  course  (when  necessary).  For  example,  the  guidance 
should  suggest  training  session  length. 

•  Course  completion  schedule.  The  maximum  time  period  in  which  the  course  must  be 
completed  should  be  specified,  and  controlled,  by  a  central  authority.  Hence,  if  the  duty 
station  initiates  a  course,  it  must  be  required  to  notify  the  central  authority. 

•  Coordination  of  training  across  the  USCG  must  be  tracked  and  controlled  by  a  central 
authority.  This  would  include  monitoring  of  duty  station  training  activities,  and 
individual  student  training  records.  Regular  reporting  of  training  activities  by  duty 
stations  would  be  required.  Software  at  the  duty  station,  to  assist  this  process,  should  be 
provided. 

•  Certification  testing  should  be  controlled  by  this  same  central  authority.  This 
includes  oversight  of  the  testing  schedule,  and  determination  of  results.  Test  results 
should  be  transmitted  to  the  relevant  parties  after  the  results  have  been  calculated  (e.g., 
student,  training  officer,  central  student  records).  The  qualification  codes  would  be 
issued  by  this  central  authority.  Note,  certification  issues  have  been  discussed  previously. 

USCG  Institute.  The  career  training  courses  currently  administered  by  the  USCG  Institute  were 
often  cited  as  a  good  example  of  nonresident  training.  This  type  of  training  has  a  mix  of  local 
and  central  control.  A  somewhat  similar  mix  of  control  was  suggested  by  a  number  of  persons, 
going  so  far  as  to  suggest  that  the  USCG  Institute  administer  all  nonresident  courses. 

5.7.2  Local  Training  Management 

Each  duty  station  should  have  one,  and  perhaps  two,  training  managers.  If  two,  their  functions 
should  be: 

•  Unit  Training  Manager.  This  is  a  management  function,  concerned  with  the 
management  of  all  training  activities  at  the  duty  station,  which  may  be  considerable  if 
nonresident  training  is  implemented  on  a  large-scale.  Examples  are: 

Coordinate  and  schedule  all  unit  training.  This  would  require  coordination  with 
persons  within  the  duty  station,  such  as  the  students’  immediate  supervisors.  It  would 
involve  scheduling  courses,  training  materials,  and  training  spaces. 

-  Coordinate  all  unit  training  personnel  and  tasking.  This  would  involve  coordination 
of  persons  involved  with  training  activities,  such  as  the  facilitator  and  others 
providing  assistance. 

-  Administer  training  tests,  as  required. 

-  Oversee  student  certification  activities. 
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-  Administer  unit  training  records.  This  would  include  management  of  the  unit’s 
training  materials  library  and  hardware  (e.g.,  training  computers),  unit  records  and 
student  records. 

-  Provide  the  primary  interface  with  the  central  training  authority.  This  may  involve 
requesting  course  materials,  coordination  of  testing  and  certification,  and  transmittal 
of  unit  and  individual  student  records. 

•  Training  Facilitator.  This  function,  for  which  multiple  people  may  required,  would 
involve  overseeing  the  detailed  training  activities  of  one  or  more  students.  Issues  related 
to  the  facilitator  are  addressed  in  the  next  subsection. 

5.7.3  Facilitator 


A  training  facilitator  is  deemed  necessary  to  work  with  each  student  during  a  nonresident  course. 
The  facilitator’s  function  would  not  be  to  train,  but  rather  to  assist  the  student  with  arrangements 
and  problems,  to  provide  general  guidance,  and  to  monitor  the  student’s  progress.  Generally,  the 
facilitator  would  not  be  expected  to  spend  a  lot  of  time  with  a  student,  but  would  conduct  at  least 
daily  monitoring  of  the  student’s  training  progress.  All  persons  interviewed  indicated  the 
importance  of  having  a  facilitator  at  each  duty  station,  who  will  interact  with  each  student. 
Several  suggestions  in  this  regard: 

•  Who.  The  facilitator  may  be  the  student’s  immediate  supervisor,  or  an  appropriate 
person  senior  to  the  student. 

•  Facilitator’s  tasks: 

Assist  the  Training  Manager  with  arrangements  for  the  student,  such  as  designating 
the  training  space,  scheduling  the  training  session  time  blocks,  and  making 
arrangements  for  special  course  requirements  (e.g.,  hands-on  training).  Over  the  long 
term,  if  nonresident  training  is  conducted  on  a  large-scale,  many  of  these  tasks  should 
become  routine  at  each  duty  station  (e.g.,  designating  the  training  location). 

-  Guide  the  student,  as  necessary.  For  example,  the  facilitator  would  provide  guidance 
for  new  staff  members  with  regard  to  the  unit’s  normal  training  practices  and 
procedures.  Guidance  might  also  be  provided  with  regard  to  special  course 
requirements,  such  as  hands-on  training. 

-  Monitor  the  student’s  progress  daily  to  identify  and  solve  training  process  problems. 

-  Oversee  the  training  materials  used  by  the  student.  These  could  include  course 
software,  manuals,  a  computer,  and  test  materials. 

•  Number  of  facilitators  needed  at  a  duty  station  will  depend  on  the  number  of  students 
being  trained,  the  amount  of  facilitator  involvement  for  a  particular  course  and  student, 
ongoing  operations  at  the  unit,  and  other  factors.  The  facilitator  function  may  be  part- 
time,  or  full-time,  depending  on  the  student  load.  As  noted  above,  the  facilitator  should 
not  be  expected  to  spend  a  lot  of  time  with  the  student  each  day,  perhaps  as  little  as  five 
minutes  per  day  to  monitor  progress.  The  time  demand,  however,  would  increase  with 
the  occurrence  of  certain  events,  such  as  student  problems  or  the  need  for  testing 
assistance. 

•  Software  tools  should  be  provided  to  assist  the  facilitator  in  monitoring  student  progress. 
The  tools  should  provide  a  quick  summary  to  the  facilitator  upon  request,  indicative  of 
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student  progress  and  alertment  of  particular  learning  difficulties.  These  are  addressed 
further  in  the  next  subsection. 

•  Formal  training  should  be  given  to  prepare  the  facilitator  for  this  important  function. 

5.8  DUTY  STATION  SUPPORT 


This  subsection  discusses  suggestions  for  support  to  be  given  to  duty  stations. 

5.8.1  Nonresident  Training  Guidelines 

USCG  personnel  at  all  levels  will  be  faced  with  a  major  change  in  responsibilities  if  nonresident 
training  is  implemented  on  a  large-scale  (i.e.,  a  large  number  of  nonresident  courses  distributed 
USCG  wide).  Although  some  training  has  always  occurred  at  duty  stations,  the  major 
responsibility  for  formal  training  has  resided  with  USCG  training  organizations.  Large-scale 
nonresident  training  will  shift  a  substantial  portion  of  this  responsibility  to  units,  and  to 
individuals  within  those  units,  including  the  students  themselves.  Guidance  is  required  at  all 
levels  within  the  USCG  to  assure  the  development  and  execution  of  an  effective  nonresident 
training  program.  The  guidelines  would  represent  the  USCG’s  policies  for  the  conduct  of 
nonresident  training,  and  assist  their  implementation  with  general  and  specific  guidance.  At  least 
three  levels  of  guidelines  are  suggested,  as  discussed  below. 

Student  Training  Guide.  A  student  training  guide  was  suggested  earlier  (See  Section  5.5.2, 
Student-Computer  Interface).  This  would  be  generic,  and  help  to  guide  the  student  in  the 
conduct  of  nonresident  training.  Course-specific  guidance  material  should  also  be  provided, 
accompanying  each  course. 

Duty  Station  Training  Guidelines.  Detailed  guidance,  on  the  establishment  and  conduct  of 
effective  training  practices  and  procedures,  should  be  developed  and  provided  to  each  duty 
station.  These  were  considered  very  important  by  many  of  the  interviewed  staff,  for  the 
establishment  of  an  effective  training  process  at  duty  stations.  The  guidelines’  content  should 
address  the  range  of  issues  affecting  the  implementation  and  conduct  of  training,  including: 

•  Nonresident  training  goals  at  duty  stations. 

•  Requirements  for  the  conduct  of  training  at  duty  stations. 

•  Types  of  training  to  be  conducted  at  duty  stations,  and  considerations  for  their 
achievement  (e.g.,  options  for  achieving  hands-on  training). 

•  How  to  establish  an  effective  training  structure  at  a  duty  station,  tailored  for  the  major 
types  of  duty  stations.  How  to  integrate  training  with  other  tasks  and  activities,  including 
prioritization  and  management. 

•  How  to  conduct  training  at  duty  stations,  including  formal  procedures  for  the  training 
process  (e.g.,  training  environment  characteristics,  training  session  length,  facilitator 
tasks). 

•  Student  monitoring  and  management. 

•  Testing  and  certification. 
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•  Good  training  practices. 

•  Coordination  with  the  District  and  other  Coast  Guard  training  organizations. 

•  Applied  examples. 

District  Guidelines.  Guidelines  should  be  developed  for  USCG  districts,  who  will  also  have  a 
training  management  role  due  to  the  interaction  of  training  and  operations  tasks.  These 
guidelines  should  be  similar  to  those  for  the  duty  stations,  but  addressing  the  issues  from  the 
district  perspective.  Importantly,  these  guidelines  should  make  commanders  aware  of  the  volume 
of  training  that  must  be  conducted,  and  the  requirements  of  a  duty  station  environment  conducive 
to  training. 


5.8.2  Software  Management  Tools 

The  amount  of  coordination  to  be  required  for  large-scale  nonresident  training  is  expected  to  be 
greater  than  that  required  under  the  current  resident  training  system,  due  to  the  distributed 
manner  in  which  training  would  be  conducted.  This  is  expected  to  place  a  greater  monitoring 
and  coordination  burden  on  the  duty  stations. 

A  set  of  software  tools  should  be  developed  to  assist  the  management  of  training  at  duty  stations. 
These  tools  would  assist  with  the  local  training  activities,  and  with  coordination  between  the  duty 
station  and  other  USCG  training  and  management  organizations.  Examples  of  support: 

•  Configuration  management  of  training  and  testing  materials  at  the  duty  station, 
including  courses,  manuals  and  hardware. 

•  Management  bookkeeping  for  all  training  and  testing  materials  at  the  duty  station. 

•  Catalogue  of  all  courses  and  materials  available  for  nonresident  training.  This  should 
include  description  of  the  training  and  certification  process,  and  materials  associated  with 
each  course. 

•  Electronic  ordering  of  courses  and  associated  materials. 

•  Student  training  progress  monitoring  (See  Section  5.7.3,  Facilitator).  This  should 
include  a  simple  quick-look  information  screen  that  can  be  accessed  by  the  facilitator,  to 
obtain  an  immediate  summary  of  the  student’s  training  progress  in  the  particular  course. 

A  basic-level  automatic  diagnostic  capability  should  obtain  the  relevant  student  course 
progress  data,  and  assemble  the  summary  screen  with  information  that  will  alert  the 
facilitator  to  student  difficulties.  Note,  this  capability  would  require  a  computer-based 
course,  and  the  student  to  be  connected  to  a  local  network  (e.g.,  if  training  at  home  on  a 
laptop  computer,  automatic  monitoring  may  not  be  feasible). 

•  Special  assistance  with  training  and  certification  activities,  such  as  the  previously 
suggested  communication  link  with  an  SME  to  conduct  a  student  certification  test. 

Special  tools  may,  or  may  not  be  required  for  these  activities. 

•  Maintenance  of  duty  station  and  student  training  records. 

•  Support  of  administrative  and  management  coordination  with  USCG  training 
organizations  external  to  the  duty  station,  including  Group  and  District  commands,  as 
necessary.  These  tools  would  collect,  assemble  and  transmit  status  information  and 
reports,  as  required.  They  are  particularly  important  to  reduce  the  duty  station 
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administrative  load  that  will  be  required  to  support  the  central  monitoring  of  training 
activities  (e.g.,  courses  conducted,  certifications  awarded,  personnel  advancement 
records). 

•  Support  of  other  training  activities  at  the  duty  station. 

5.9  COMMAND  SUPPORT 

A  particularly  important  issue  identified  by  most  persons  interviewed  is  the  need  for  command 
buy-in  of  nonresident  training,  meaning  command  at  all  levels  within  the  USCG,  from  the 
admiral-level  to  the  duty  station  commander.  The  commands  are  viewed  as  focused  on  their 
primary  operational  mission  goals,  which  for  field  units  usually  does  not  include  training. 
Although  it  was  felt  that  commands  view  training  as  important,  they  also  view  operational  needs 
as  taking  priority  over  training  needs  at  the  field  units.  Although  this  is  certainly  a  reasonable 
view,  since  training  seldom  has  critical  immediate  needs  and  can  easily  be  postponed  to  a  more 
convenient  time,  it  takes  place  in  a  system  largely  dependent  on  resident-based  training  for 
development  of  critical  skills  and  knowledge.  If  a  large-scale  conversion  to  nonresident  training 
is  implemented,  operations  practices  that  are  disruptive  to  the  training  process  at  duty  stations 
(i.e.,  which  are  common  today)  may  have  a  substantial  adverse  impact  on  the  effectiveness  and 
efficiency  of  nonresident  training.  And,  in  turn,  these  practices  may  have  a  similar  adverse 
impact  on  the  long  term  operational  proficiency  of  units. 

As  suggested  earlier,  for  nonresident  training  to  succeed,  the  “training  task”  must  be  elevated  to  a 
level  roughly  equivalent  to  other  operational  tasks  in  the  view  of  commands,  and  scheduled 
accordingly.  The  requisite  training  must  be  conducted  in  a  duty  station  environment  that  is 
conducive  to  effective  training.  The  persons  interviewed  generally  felt  that  command  buy-in  at 
all  levels  would  be  necessary  to  assure  proper  emphasis  is  given  to  the  training  process  at  duty 
stations.  As  one  person  stated,  a  philosophical  culture  change  within  the  USCG  will  be 
necessary  to  achieve  an  environment  at  duty  stations  that  will  facilitate  training. 

Several  suggestions  to  achieve  command  support  were: 

•  Nonresident  training  guidelines,  as  suggested  above. 

•  Strong  emphasis  on  the  need  for  good  training  practices  from  all  levels  of  the  USCG 
(e.g.,  Headquarters). 

•  Insert  a  major  module  in  the  PCO/PXO  course  addressing  nonresident  training.  This 
module  should  focus  on  the  importance  of  nonresident  training,  characteristics  of  a  duty 
station  environment  conducive  to  training,  and  how  to  manage  and  conduct  nonresident 
training. 

•  Assurances  that  effective  training  can  be  achieved  at  duty  stations,  if  the  appropriate 
training  practices  are  implemented. 

•  Formal  internal  USCG  marketing  effort  should  be  developed  for  nonresident  training, 
to  generate  support.  This  marketing  should  be  specifically  aimed  at  commanders,  at  all 
levels.  The  reasoning  was  that,  if  the  commander  supports  nonresident  training,  the  staff 
will  also. 
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•  Student  critique  administered  after  every  course,  containing  questions  about 
command  support  and  training  practices.  This  would  provide  early  detection  of 
problems,  and  enable  timely  corrective  action. 

5.10  HARDWARE  ISSUES 

The  hardware  issues  addressed  during  the  interviews  focused  on  CBT,  similar  to  the  AN/WSC-3 
CBT  course.  The  issues  and  suggestions  are  discussed  in  this  subsection. 

5.10.1  Local  Versus  Central  Control  of  Training  Computers 

This  issue  is  relevant  only  if  laptop  computers  are  used  for  training.  Local  control  of  computers 
would  involve  providing  a  set  of  training  computers  to  all  duty  stations,  for  the  primary  purpose 
of  conducting  nonresident  training.  The  advantage  of  local  control  would  be  having  hardware 
always  available  for  training.  The  time  delays,  cost  and  other  requirements  associated  with 
shipping  computers  from  a  central  repository  would  not  be  required,  resulting  in  a  cost  savings. 
On  the  other  hand,  local  control  would  require  hardware  management  tasks,  such  as 
accountability  and  repair.  Most  persons  preferred  having  local  control. 

Central  control  of  computers  would  compliment  a  central  repository  for  distribution  of  the 
complete  course  package  to  each  student,  at  the  time  the  course  is  to  be  conducted  (See  Section 
5.7.1,  Duty  Station  and  Central  Control  Issues).  For  example,  the  course  could  be  distributed 
pre-installed  on  the  laptop  computer,  similar  to  the  procedure  used  for  the  AN/WSC-3  CBT 
course.  This  approach  would  be  expected  to  reduce  the  administrative  load  at  duty  stations,  but 
also  reduce  their  flexibility.  Its  cost  would  likely  be  higher,  due  to  shipping  (i.e.,  shipping  costs, 
packaging,  breakage,  inventory).  Central  control  was  generally  considered  acceptable,  although 
not  preferred,  by  most  persons.  Some  individuals,  however,  did  prefer  this  approach,  citing  less 
administrative  work  for  smaller  units. 

5.10.2  Laptop  Versus  Desktop  Computers 

This  issue  would  be  relevant  only  if  the  training  computers  were  permanently  located  at  the  duty 
stations,  since  continually  shipping  desktop  computers  would  not  be  a  viable  option.  Some 
individuals  preferred  laptops,  while  others  preferred  desktops.  The  issues  were: 

•  Laptop.  Flexibility  was  cited  as  the  primary  advantage,  enabling  the  student  to  go  to  a 
desired  training  location  with  the  computer  (e.g.,  home,  vacant  office  at  the  unit,  office  on 
the  pier,  radio  room  for  certification  testing).  Survivability  was  also  cited,  although  this 
may  not  be  accurate  if  desktop  computers  were  not  moved.  On  the  other  hand,  a  laptop 
would  be  much  more  susceptible  to  pilfering  (pilfering  of  laptop  computers  was  noted  as 
a  problem  on  two  U.S.  Navy  ships  visited  by  project  team  members).  And,  the  cost  of 
appropriately  configured  laptop  computers  would  likely  be  greater  than  equivalent 
desktop  computers. 

•  Desktop.  Computer  system  robustness  was  cited  as  the  primary  advantage.  This 
included  suggestions  that  the  person-computer  interface  (e.g.,  larger  display,  mouse)  may 
be  better  for  long  courses  and  detailed  manual  manipulations,  and  have  greater 
compatibility  with  other  hardware  and  software.  Pilfering  would  be  a  minor  issue.  The 
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primary  disadvantage  cited  was  inflexibility  for  remote  training  (e.g.,  training  at  home). 
The  greater  space  requirement  for  desktop  computers  was  also  noted  as  a  disadvantage. 

Laptop  computers  appear  preferable  to  desktop  computers,  on  the  basis  of  some  of  the 
implementation  suggestions,  such  as  allowing  flexibility  with  the  student’s  training  location,. 
Perhaps  the  best  approach  would  be  to  use  a  mix  of  laptop  and  desktop  computers  at  each  unit. 

5.10.3  Use  of  Operational  Workstations  for  Training 

The  anticipated  availability  of  USCG  Standard  Workstation  III  (CGSWIII)  computers  at  duty 
stations  was  expected  to  vary,  with  some  units  receiving  a  relatively  large  number  and  others 
receiving  few.  The  supervisory  persons  interviewed  indicated  that  it  would  be  feasible  to  use 
their  CGSWIII  computers  for  training,  but  on  a  part-time  basis  only.  Consequently,  it  was  also 
suggested  that  the  operational  workstations  would  have  to  be  augmented  by  training  computers  to 
address  large-scale  nonresident  CBT,  due  to  insufficient  numbers  of  CGSWIII  computers  to 
support  all  likely  training  needs.  Nonresident  CBT,  therefore,  would  be  conducted  using  a  mix 
of  operational  computers  and  dedicated  training  computers.  Consequently,  it  was  suggested  that 
some  number  of  training  computers  would  have  to  be  purchased  for  each  duty  station  to  support 
nonresident  CBT. 

The  computer  systems  used  for  CBT  would  likely  require  a  multimedia  configuration.  Although 
most  units  had,  or  expected  to  receive,  some  multimedia  computers,  not  all  of  CGSWIII 
computers  were  so  configured  at  the  time  interviewed.  This  would  place  additional  constraints 
on  the  availability  of  operational  computers  for  training  duty. 

5.11  QUALITY  ASSURANCE 

Large-scale  nonresident  training  will  require  a  greater  degree  of  quality  assurance  than  resident 
training,  due  to  the  much  wider  dispersion  of  training  responsibility.  Persons  interviewed 
generally  felt  that  the  USCG  will  need  some  amount  of  centralized  control  on  the  training 
process  conducted  at  duty  stations,  although  they  cautioned  against  reducing  the  inherent 
flexibility  of  nonresident  training  at  duty  stations.  The  control  suggested  was: 

•  Provide  guidance  for  the  establishment  and  conduct  of  an  effective  training  process  at 
duty  stations,  but  allow  flexibility.  This  has  been  discussed  in  several  of  the  preceding 
sections. 

•  Provide  centralized  monitoring  of  nonresident  training  to  assure  that  the  training  is 
being  properly  conducted,  and  to  identify  problem  areas  for  early  corrective  action.  This 
may  include  sampling  the  training  effectiveness  of  nonresident  courses,  by  empirically 
evaluating  selected  courses  each  year.  Several  related  suggestions  were  made  earlier, 
such  as  a  student  questionnaire  after  completion  of  every  course. 

•  Proactively  evaluate  a  subset  of  courses  each  year  to  determine  if  they  require 
upgrading.  Resident  courses  are  regularly  improved  by  their  instructors,  as  operational 
practices  and  requirements  change,  and  as  improved  training  strategies  become  available. 
Over  several  years,  such  evolutionary  changes  can  be  substantial.  A  CBT  course,  on  the 
other  hand,  will  not  change  unless  there  is  proactive  intervention.  To  maintain  an 
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effective  nonresident  training  program,  courses  must  be  periodically  evaluated,  and 
changed  when  necessary. 

5.12  OTHER  NONRESIDENT  TRAINING  ISSUES 


Several  additional  issues  indirectly  related  to  nonresident  training  are  discussed  in  this 
subsection. 


5.12.1  Other  Electronic  Media  Concepts  for  Duty  Stations 

Opinions  were  solicited  regarding  electronic  technical  manuals  and  just-in-time  training  (i.e., 
training  provided  in  CBT  form,  in  association  with  an  electronic  tech  manual,  at  the  time  a 
particular  non-critical  task  is  to  be  performed  on  the  job).  All  persons  were  supportive  of  both 
concepts. 

5.12.2  Classified  Materials 

Only  some  of  the  duty  stations  visited  were  capable  of  storing  classified  materials.  The  persons 
in  those  units  indicated  that  the  handling  and  storage  of  classified  materials  for  training  would 
not  present  any  new  problems,  as  long  as  the  training  that  accessed  classified  material  was 
conducted  on  the  premises.  Such  training  would  not  be  feasible  at  a  non-cleared  site,  such  as  the 
student’s  home,  a  potential  limitation. 

An  additional  problem  arises  if  the  classified  material  is  computer-based.  A  classified  computer 
would  be  required  in  addition  to  a  cleared  space  for  conduct  of  the  training. 

5.12.3  Pipeline  Training 

Many  individuals  are  not  pleased  with  the  way  pipeline  training  is  conducted  today,  in  which  a 
student  is  sent  to  a  course  in  preparation  for  the  next  duty  station,  while  still  assigned  to  the 
current  duty  station.  They  felt  this  places  added  burdens  on  the  student’s  current  duty  station. 
With  this  situation  in  mind,  considering  the  time  savings  often  associated  with  CBT  courses,  and 
the  nearby  partial  availability  of  a  student  engaged  in  training  at  the  duty  station,  all  persons 
favored  nonresident  courses  for  satisfying  pipeline  training  requirements.  In  fact,  a  number  of 
persons  suggested  that  nonresident  training  could  be  a  key  partial  solution  to  the  “pipeline 
training  problem.” 

Some  potential  problems  associated  with  nonresident  courses  for  pipeline  training  were  noted 
earlier,  particularly  in  regard  to  achieving  necessary  hands-on  training  (See  Section  5.5.4,  Hands- 
On  Training). 

Several  of  the  persons  interviewed  felt  that  pipeline  training  would  be  better  provided  at  the 
student’s  new  duty  station,  or  during  the  period  of  transition  between  duty  stations. 
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5.12.4  Minimum  Crewing 


Opinions  were  received  both  for  and  against  minimum  crewing.  On  the  other  hand,  everyone 
spoken  with  about  this  issue  gave  the  opinion  that  nonresident  training  will  be  a  definite  benefit 
to  minimum-crewed  vessels.  The  two  basic  reasons  were: 

•  Overall  time  savings  to  the  crew,  due  to  the  typically  reduced  training  time  associated 
with  CBT  courses.  This  would  translate  into  the  student  being  available  for  more  work, 
and  a  correspondingly  smaller  burden  for  the  rest  of  the  crew. 

•  Nearby  availability  of  the  student  during  the  period  of  training  (i.e.,  either  at  the  duty 
station,  at  home,  or  elsewhere  nearby).  The  student  would  be  available  to  assist  in  case  of 
an  emergency. 

5.12.5  Learning  Centers 

Learning  centers  could  be  provided  shoreside,  near  many  operating  units,  to  furnish  appropriate 
training  spaces.  The  learning  centers  would  consist  of  one  or  more  rooms  devoted  to  training. 

The  rooms  could  have  computers,  or  just  table-top  space.  They  would  be  available  for  training 
students  from  any  unit.  Each  of  the  two  Engineering  Support  Units  visited  had  a  room  with 
desktop  computers,  devoted  to  training. 

A  similar  learning  center  space  could  be  designated  at  each  shoreside  duty  station,  in  accordance 
with  the  unit’s  size  and  other  factors.  The  learning  center  would  facilitate  nonresident  CBT  by 
reducing  problems  associated  with  the  training  environment.  Cost  would  be  the  primary  concern. 
The  learning  center  was  also  discussed  earlier,  in  Section  5.3.4,  Potential  Solutions  to  Training 
Environment  Problems. 

Mixed  views  were  obtained  about  locating  a  learning  center  shipboard.  Many  people  felt  it  was 
feasible  on  larger  cutters,  especially  if  nonresident  CBT  was  to  be  implemented  on  a  large  scale. 
The  space  limitations  on  smaller  vessels  may  prohibit  this  type  of  space.  One  suggestion  was  to 
make  the  messroom  multi-purpose,  by  fitting  computers  into  the  tables.  It  could  then  serve  as  a 
learning  center  part-time.  The  navy  has  learning  resource  centers  on  some  ships,  and  at  shore 
bases.  On  the  two  U.S.  Navy  ships  visited  by  project  team  members,  the  learning  resource  center 
was  used  primarily  for  morale  purposes  (e.g.,  email  link  to  family,  entertainment).  The  CBT 
courses  that  were  conducted  on  that  ship,  were  done  so  on  the  computers  in  the  various 
workspaces. 

5.13  CONVERSION  TO  NONRESIDENT  TRAINING 

This  section  addresses  the  conversion  of  course  to  a  nonresident  training  basis,  and  discusses  the 
need  for  a  nonresident  training  implementation  plan. 

5.13.1  Course  Conversions 


A  variety  of  suggestions  were  received  for  courses  that  would  be  good  candidates  for  conversion 
to  nonresident  training.  In  addition,  the  suggestion  was  made  by  several  persons  that  much  of  the 
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current  nonresident  training  should  be  converted  to  CBT  courses.  This  suggestion  included 
correspondence  courses,  and  general  safety  and  other  qualification  training  received  when 
reporting  to  a  new  duty  station.  The  suggestions  were: 


Unit  training,  such  as  shipboard  qualifications,  and  unit  safety  qualifications. 
All  correspondence  courses.  Personal  advancement  courses. 

All  short  courses. 

Most  C-Schools. 

General  shop  skills  and  knowledge. 

All  equipment  at  the  duty  station. 

Pipeline  training. 

Administrative  processes. 

Divisional  and  operational  processes  at  the  unit,  such  as: 

Piloting  and  navigation 

Communications,  radio  telephone 

Getting  underway 

Watchstanding 

Rules  of  the  Road 

Buoy  deck 

Signaling 

Specific  equipment  or  course: 

Global  Positioning  System  (GPS)  receivers,  transceivers 

Loud  hailer 

INMARSAT 

64  radar  course,  first  2  weeks 
Aids  to  navigation 
Search  and  rescue  (SAR) 

AN/WSC-3 

Electronic  fundamentals,  A-School 
Test  equipment 
General  military  training. 


It  was  suggested  that  certain  courses  should  be  tailored  to  the  particular  ship,  such  as  a  damage 
control  CBT  course. 


Some  courses  may  be  wholly  converted  into  a  nonresident  CBT  form,  as  was  accomplished  with 
the  AN/WSC-3  course.  Some  long  courses  may  require  conversion  into  two  or  more  CBT 
courses,  due  to  their  length.  Some  courses  may  require  a  partial  conversion  to  a  CBT  format, 
while  retaining  a  resident  course  component.  In  these  latter  courses,  either  the  nonresident  CBT 
course  or  the  new  resident  course  may  become  the  prerequisite  for  the  other. 

It  should  be  expected  that  large-scale  implementation  of  nonresident  training  may  necessitate 
changes  to  many  USCG  courses,  even  those  not  directly  affected  by  conversion.  Such  changes 
may  be  the  most  cost-effective  way  of  achieving  the  resident-based  training  requirements 
associated  with  the  courses  that  have  been  partially  converted.  For  example,  the  A-School 
curriculum  might  be  modified  to  include  more  hands-on  training,  and  thus  facilitate  conversion 
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of  certain  courses  to  CBT  form.  Also,  the  resident-based  parts  of  certain  partially  converted 
courses  may  be  combined  into  a  single  new  course. 

5.13.2  Nonresident  Training  Implementation  Plan 

A  detailed  implementation  plan  should  be  developed  to  guide  transition  from  resident  to 
nonresident  training,  if  large-scale  nonresident  training  is  to  be  implemented.  A  phased 
transition  process  is  suggested,  to  allow  a  gradual  growth  of  user  familiarity  and  trust, 
necessitating  a  well  thought-out  plan.  Most  persons  interviewed  stressed  that  simply 
implementing  nonresident  training  within  the  current  USCG  structure  would  not  work.  Many 
issues  need  careful  consideration,  and  pragmatic  solutions.  The  implementation  plan  would  be 
the  blueprint  for  converting  the  USCG’s  training  system,  representing  the  implementation 
decisions.  Several  suggestions  follow. 

The  many  issues  addressed  in  this  document  show  that  large-scale  nonresident  training  will 
require  some  re-organization  of  the  Coast  Guard’s  infrastructure,  affecting  both  operations  and 
training.  The  required  decisions  include  organizational  changes  to  establish  a  system  for 
managing  resident  and  nonresident  training,  including  the  necessary  checks  and  balances  to 
assure  that  effective  training  is  achieved  on  a  regular  basis  at  all  duty  stations.  It  includes  a 
system  to  convert,  modify  and  develop  the  resident  and  nonresident  courses  and  materials;  to 
evaluate  and  upgrade  courses  when  necessary.  And  it  includes  the  structure  at  the  duty  station  to 
manage  and  conduct  training  on  a  daily  basis. 

The  Implementation  Plan  should  generate  detailed  trade-off  information  about  the  most 
promising  approaches  to  the  many  relevant  issues.  Strawmen  policies  and  procedures,  when 
appropriate,  should  be  developed  and  field-tested.  This  process  should  work  toward  the  second 
phase,  the  detailed  plan  for  implementation.  The  plan  must  address  the  issues  identified  in  this 
report,  and  those  subsequently  identified.  It  should  also  address  the  USCG  organizations  that 
will  play  major  roles  in  the  development,  implementation  and  later  conduct  of  nonresident 
training. 

Nonresident  training  should  be  evolved  carefully,  to  assure  that  the  converted  courses  are 
effective,  and  to  guide  the  development  of  effective  training  practices  at  duty  stations.  An 
important  part  of  the  early  implementation  phase  should  be  to  validate  nonresident  courses,  to 
demonstrate  the  effectiveness  of  nonresident  training  in  different  operational  settings,  and  for 
different  types  of  training  objectives.  It  is  imperative  that  the  courses  converted  be  of  high 
quality,  and  effective.  The  marketing  of  nonresident  training  to  all  levels  within  the  USCG 
should  be  an  important  objective  during  the  early  phase. 

The  later  phases  should  accelerate  the  course  conversions,  to  take  advantage  of  the  developing 
infrastructure  and  readiness  for  nonresident  training  at  the  duty  stations.  Early  lessons  learned 
should  be  addressed  by  adapting  the  developing  system.  And,  the  major  elements  of  the  revised 
infrastructure  should  be  implemented.  A  well  thought-out  plan  is  essential  to  development  and 
implementation  of  an  effective  resident/nonresident  training  system. 
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6 


CONCLUSIONS  AND  RECOMMENDATIONS 


6.1  CONCLUSIONS 

Operation  and  maintenance  training  delivered  at  USCG  duty  stations  via  nonresident  CBT  can  be 
as  effective  as  instructor-led  training  (courses)  conducted  at  resident  training  centers.  The  results 
of  this  study  also  show  that  CBT  courses  delivered  to  students  at  resident  training  centers  can  be 
as  effective  as  instructor-led  courses  conducted  at  resident  training  centers.  Each  of  the  three 
training  methods  investigated  in  this  study  (CBT  Duty  Station,  CBT  Remote  (Resident),  and 
Resident  (Instructor-led  resident))  were  effective  in  imparting  knowledge  to  the  students. 
Significant  training  gains  were  measured  for  each  method. 

A  computer-based  version  of  the  AN/WSC-3(v)7  UHF  Transceiver  Maintenance  Course  was 
effectively  conducted  at  13  USCG  duty  stations,  including  cutters  and  shore-stations  on  the  East, 
Gulf,  and  West  coasts.  This  training  was  found  to  be  equal  in  effectiveness  to  the  instructor-led 
version  of  the  course  conducted  at  a  resident  training  center.  Commanders,  students,  and  other 
staff  members  interviewed  at  participating  duty  stations,  strongly  support  implementation  of 
nonresident  training  (e.g.,  CBT,  IVT)  on  a  large  scale  within  the  USCG.  However,  successful 
implementation  of  nonresident  training  will  require  a  substantial  effort  by  the  USCG  to  develop 
and  execute  a  training  process  at  duty  stations  that  will  be  effective.  These  and  related 
conclusions  are  presented  in  this  subsection  of  the  report. 

Nonresident  CBT  at  Duty  Stations 

Substantially  reduced  training  time  is  a  potentially  major  benefit  of  nonresident  CBT  conducted 
at  duty  stations,  in  comparison  with  traditional  instructor-led  resident  training.  The  nonresident 
CBT  version  of  the  AN/WSC-3  course  required  an  average  70%  less  training  time  than  the 
equivalent  instructor-led  resident  course.  This  advantage  was  due  to  CBT  methodology,  and  not 
the  training  location;  similar  reductions  in  training  time  were  achieved  with  the  CBT  course 
conducted  at  the  training  center. 

Students  identified  strengths  and  weaknesses  of  nonresident  CBT  conducted  at  duty  stations. 

The  most  beneficial  parts  of  the  AN/WSC-3  CBT  course  identified  were  characteristics  inherent 
to  CBT  (e.g.,  self-paced  training),  advantages  of  training  at  their  duty  stations,  and  less  training 
time.  They  strongly  approved  of  the  AN/WSC-3  CBT  course’s  user  interface.  Additional 
practice  problems  were  the  most-often-cited  need  for  improvement.  Other  needs  cited  included 
additional  theory,  and  hands-on  training  at  their  duty  station,  rather  than  at  the  training  center. 

The  equivalent  effectiveness  of  nonresident  CBT  and  resident  training  refers  to  a  potential  of 
equality,  rather  than  a  certainty  of  equality.  The  actual  long-term  effectiveness  of  nonresident 
CBT  across  the  USCG  will  depend  heavily  on  the  manner  in  which  the  training  is  conducted  at 
duty  stations  (i.e.,  the  specific  training  process  at  each  duty  station).  The  experiment  exerted  a 
high  degree  of  control  over  the  one-time  training,  with  participants  informed  of  the  importance  of 
their  thoughtful  participation.  A  large-scale  nonresident  training  process  will  have  to  create  an 
environment  conducive  to  effective  training  at  duty  stations. 
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Personnel  Support  of  Nonresident  Training  at  Duty  Stations 


Duty  station  staffs,  at  command  and  supervisory  levels,  who  have  had  experience  with 
nonresident  CBT,  strongly  support  the  implementation  of  training  at  their  duty  stations.  They 
feel  that  nonresident  training  (e.g.,  CBT,  IVT)  can  definitely  succeed  in  the  USCG,  and  will  be  a 
major  benefit  to  duty  stations.  The  primary  reason  for  this  support  is  two-fold:  1)  nonresident 
training  gives  the  duty  station  much  greater  flexibility  in  training  its  staff;  and  2)  nonresident 
training  is  less  of  a  time  burden  on  the  duty  station,  resulting  in  greater  availability  of  staff  to 
assist  with  normal  and  emergency  activities.  Nonresident  training  is  viewed  by  duty  station  staff 
as: 

•  Having  less  adverse  impact  than  resident  training  on  the  overall  team’s  ability  to  perform 
its  mission;  resulting  in  greater  staff  availability;  minimizing  unit  disruptions;  easing  unit 
staffing  problems;  and  resulting  in  increased  training  efficiency. 

•  Assisting  their  general  unit  training  requirements. 

•  A  partial  solution  to  problems  associated  with  minimal-crewed  vessels. 

•  Helping  the  pipeline  training  problem. 

Students  are  concerned  about  the  need  to  develop  a  duty  station  environment  that  is  conducive  to 
training,  if  nonresident  training  is  to  succeed.  Students  were  generally  favorable  toward 
nonresident  training,  although  a  range  of  opinions  was  expressed.  There  was  an  indication  in  the 
data,  although  not  a  significant  finding,  that  the  students  were  slightly  more  positive  toward 
training  at  the  resident  training  centers.  This  indication  was  concluded  to  infer  that,  today,  the 
training  center  environment  is  more  conducive  to  training  than  that  of  the  duty  station.  This 
conclusion  further  suggests  the  need  to  carefully  construct,  and  maintain,  the  duty  station 
environment  in  a  manner  favorable  to  the  achievement  of  training.  The  students  cited  specific 
characteristics  of  the  duty  station  training  environment  that  must  be  improved  to  assure  an 
adequate  training  process.  Most  important  among  these  were  the  need  for  elimination  of  training 
interruptions,  combined  training  and  work  prioritizing  and  scheduling,  and  increasing  command 
emphasis  on  training. 

Implementation  of  Nonresident  Training  at  Duty  Stations 

Duty  station  staff,  in  addition  to  their  support  of  nonresident  training  (e.g.,  CBT,  IVT),  also 
believe  existing  obstacles  must  be  overcome  with  careful  organization,  and  development  and 
execution  of  the  training  process.  Essential  to  success  is  duty  station  flexibility  in  directing  the 
training.  Many  implementation  issues  are  identified  in  the  report,  with  alternative  suggestions 
given  for  their  resolution.  In  summary,  it  is  concluded  that  an  effective  training  process  at  duty 
stations  requires  the  following: 

•  Strong  overt  command  support  for  training  should  be  given,  at  all  levels  across  the 
USCG. 

•  Sufficient  time  should  be  provided  daily,  during  normal  work  hours,  for  the  conduct  of 
training.  Training  should  be  scheduled  like  any  other  work  task. 
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•  Interruptions  must  not  be  allowed  while  students  are  actively  engaged  in  training,  except 
for  real  emergencies. 

•  The  area  and  room  in  which  a  student  is  training  must  have  an  environment  conducive  to 
learning,  such  as  minimizing  distractions. 

•  The  training  process  and  procedures  must  provide  the  unit  flexibility  in  scheduling  and 
conducting  training  in  accordance  with  their  particular  situation.  This  includes  a  strong 
organizational  structure  at  the  duty  station,  and  at  other  levels  in  the  USCG,  to  manage 
training. 

•  Selection  of  courses  for  nonresident  training  conversion  on  the  basis  of  a  strong  potential 
for  their  effective  implementation  in  CBT,  or  other  media  conducive  to  duty  station 
training.  This  means  the  skills  and  knowledge  to  be  trained,  and  the  strategies  to  be 
employed,  favor  nonresident  methods  (e.g.,  CBT,  IVT). 

•  Development  of  effective  nonresident  training  courses. 

Student  motivation  is  an  important  factor  to  the  success  of  nonresident  training.  Suggestions  for 
dealing  with  this  issue  include  student  supervision  and  incentives. 

The  facilitators  participating  in  the  duty  station  training  did  an  adequate  job  working  with  the 
students.  Student  opinions  about  the  assistance  provided  by  their  facilitator  varied  widely.  This 
underscores  the  above  conclusion  about  the  need  to  carefully  construct  and  maintain  a  duty 
station  environment  favorable  to  training,  including  strong  support  at  all  levels  of  command. 

Hands-on  training  with  actual  equipment,  to  augment  the  CBT  medium,  is  potentially  important 
to  the  success  of  many  nonresident  CBT  courses.  Effective  ways  of  achieving  hands-on  training 
should  be  decided  by  each  duty  station,  with  guidance  provided  by  Headquarters. 

A  variety  of  hardware  issues  exist.  These  include: 

•  Local  versus  central  control  of  training  computers 

•  Laptop  versus  desktop  computers 

•  Use  of  unit  CGSWIII  computer  systems  for  training.  CGSWIII  computers  were 
anticipated  to  be  available  for  training  use  part-time.  However,  large  scale  course 
conversions  to  nonresident  CBT  may  require  additional  multimedia  computer  systems  at 
many  USCG  units,  or  supporting  sites. 

A  centralized  quality  assurance  process  should  be  developed  as  a  check-and-balance,  to  assure 
that  an  adequate  level  of  training  effectiveness  is  developed  and  maintained  at  all  duty  stations. 
This  process  should  also  be  responsible  for  maintaining,  modifying  and  upgrading  courses  as 
necessary. 

Nonresident  training  involving  classified  material  will  require  special  consideration,  for 
information  handling  and  storage,  training  hardware  and  training  space.  These  requirements  are 
not  currently  met  at  all  duty  stations. 

It  is  likely  that  a  proportion  of  the  courses  currently  taught  at  training  centers,  and  also  modules 
within  courses  if  not  the  whole  course,  may  be  effectively  converted  to  nonresident  training.  The 
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suitability  of  a  course  for  nonresident  training  will  depend  on  the  training  objectives,  course 
content,  training  methods,  and  other  factors. 

The  method  of  student  certification  (e.g.,  awarding  a  qualification  code),  after  completion  of  a 
nonresident  course,  is  an  important  issue.  Suggestions  for  dealing  with  this  issue  were  identified. 

A  detailed  transition  plan  is  required  to  guide  the  implementation  of  nonresident  training  across 
the  USCG.  This  plan  will  require  substantial  additional  research  to  resolve  many  of  the 
identified  issues,  as  well  as  other  issues  pertinent  to  the  establishment  of  an  effective  training 
process  at  duty  stations. 

Investigation  Methodology 

A  high  degree  of  consistency  in  the  results  was  obtained  across  the  variety  of  data  collected  and 
analyses  performed.  This  indicates  good  reliability  in  the  data,  and  high  confidence  in  the 
findings. 

The  highly  structured  and  applied  hands-on  testing  methodology,  developed  under  this 
investigation,  was  suitable  for  the  evaluation  of  ANAVSC-3  operation  and  maintenance 
proficiency.  The  hands-on  testing  methodology,  although  developed  specifically  for  this  study, 
has  a  generic  structure  that  may  be  useful  to  evaluation  of  operation  and  maintenance  skills  in 
other  domains,  and  for  other  purposes.  The  structure  is  based  on  generic  assessment  techniques 
and  characteristics,  and  has  both  generic  and  ANAVSC-3  specific  content.  This  conclusion  was 
reached  after  application  of  the  hands-on  test  to  evaluate  students  during  the  performance  of  235 
troubleshooting  problems.  The  independent  evaluators’  assessments  agreed  well  with  each  other, 
and  their  overall  judgments  agreed  well  with  the  detailed  performance  scoring. 

6.2  RECOMMENDATIONS 


The  USCG  should  proceed  with  the  implementation  of  nonresident  training  at  duty  stations.  This 
will  require  selection  of  courses  to  convert,  determining  the  appropriate  cost-effective  media  mix 
(e.g.,  CBT,  IVT,  correspondence)  for  each  course  selected,  conversion  of  the  courses,  and 
establishment  of  an  effective  training  process  at  duty  stations.  Establishing  an  effective  training 
process  will  be  complex,  requiring  careful  investigation  of  many  issues,  implementation  of 
prototype  courses  and  processes,  and  making  pragmatic  decisions  to  create  a  duty  station 
environment  conducive  to  training. 

The  many  issues  associated  with  large-scale  implementation  of  nonresident  training  are  expected 
to  require  some  re-organization  of  the  USCG’s  infrastructure,  affecting  both  operations  and 
training.  These  should  include  organizational  changes  for  managing  resident  and  nonresident 
training,  and  the  necessary  checks-and-balances  to  assure  that  effective  training  is  achieved  on  a 
regular  basis  at  duty  stations.  The  establishment  of  nonresident  training  at  duty  stations  should  be 
a  cautious  phased  process,  rather  than  a  bulk  implementation. 

The  selection  of  resident  courses  for  conversion  should  move  forward.  These  should  be 
prioritized  on  the  basis  of  potential  for  successful  implementation,  considering  the  many  relevant 
factors.  Development  should  begin  for  several  of  the  highest  priority  courses  only.  Many 
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recommendations  have  been  made  identifying  courses  to  convert.  These  are  presented  in  the 
body  of  the  report,  under  Section  3.3.3,  Application  of  Nonresident  Training,  and  Section  5.13.1, 
Course  Conversions. 

Special  emphasis  should  be  placed  on  assuring  that  the  initial  courses  converted  to  a  nonresident 
basis,  are  as  effective  as  the  resident  courses  they  replace.  These  courses  will  provide  the  very 
important  role  model  for  subsequent  course  development.  More  importantly,  they  will  provide 
the  foundation  for  initial  acceptance,  or  rejection,  of  nonresident  training  across  the  USCG.  And, 
these  initial  courses  will  be  central  to  development  of  the  training  process  and  procedures  at  duty 
stations. 

The  students  taking  the  AN/WSC-3  CBT  course  made  a  variety  of  improvement 
recommendations,  which  are  addressed  in  Section  3.3,  Student  Reaction  to  the  AN/WSC-3  CBT 
Course.  Most  notable  of  these  was  the  addition  of  more  practice  problems  in  the  CBT  medium. 

Many  suggestions  for  establishment  of  an  effective  training  process  at  duty  stations  were  made 
during  the  interviews,  and  are  addressed  in  the  report  under  Section  5,  Duty  Station 
Implementation  Analysis  Findings.  Examples  of  important  recommendations  are: 

•  Strong  overt  command  support  for  duty  station  training,  at  all  levels  across  the  USCG, 
must  be  given  and  demonstrated. 

•  Duty  station  training  process  and  procedures  should  consider  the  following: 

-  Daily  blocks  of  training  time  during  normal  work  hours  should  be  made  available  to 
students  when  engaged  in  a  nonresident  course.  These  should  be  scheduled  along 
with  work  tasks,  to  ensure  sufficient  training  time  on  a  regular  basis. 

Appropriate  training  spaces  should  be  designated  at  each  duty  station.  These  may 
include  rooms  at  the  facility  (e.g.,  compartment  on  a  cutter),  nearby  rooms  (e.g., 
pierside  office),  and  the  student’s  home.  The  training  space,  which  may  be  identified 
on  an  individual  student  basis,  should  provide  an  environment  conducive  to  learning 
(e.g.,  no  interruptions  or  distractions). 

-  A  training  person  should  be  designated  to  oversee  the  management  of  training  at  each 
duty  station.  A  facilitator  should  be  designated  to  monitor  and  assist  each  student 
taking  a  course. 

The  required  training  process  characteristics  must  not  reduce  the  duty  station’s 
flexibility  to  control  and  conduct  training  in  accordance  with  their  particular  situation. 

•  Student-help  hotline  -  is  recommended  in  association  with  each  course,  to  provide  all 
duty  station  students  with  immediate  access  to  a  subject  matter  expert,  to  answer 
technical  questions. 

•  Standard  student-computer  interface  -  should  be  specified  to  assure  commonality 
across  all  nonresident  courses.  This  interface  requirement  should  also  permit  evolution  in 
accordance  with  commercial  computer  products. 

•  Hands-on  training  -  may  be  necessary  to  support  certain  courses.  Each  duty  station 
should  decide  the  best  method  for  achieving  such  requisite  training,  in  accordance  with 
guidance  provided  by  a  central  training  authority,  based  on  the  particular  duty  station  and 
student  situations. 
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A  section  should  be  added  to  the  USCG’s  PCO/PXO  Course,  addressing  the  establishment  and 
conduct  of  an  effective  training  process  at  duty  stations. 

An  internal  USCG  marketing  effort  should  be  established,  to  inform  personnel  at  all  levels  about 
nonresident  training.  A  variety  of  opinions,  often  strong,  exist  regarding  the  advantages  and 
disadvantages  of  training  at  duty  stations.  As  noted  in  the  report,  several  persons  interviewed 
expressed  initial  negative  opinions  about  the  AN/WSC-3  CBT  course  and  this  type  of  training, 
but  later  changed  their  opinions  after  experiencing  the  training  first-hand. 

Software  tools  should  be  developed  for  the  duty  station,  to  support  training  management, 
monitoring  and  oversight  of  student  training,  and  administrative  coordination  with  the  central 
training  authority. 

Guidelines  should  be  developed  for  duty  stations,  providing  guidance  on  how  to  develop  and 
conduct  an  effective  training  process.  They  should  not  limit  the  duty  station’s  flexibility,  but 
rather  should  provide  help  in  establishing  a  training  process  tailored  to  the  unit’s  particular 
situation.  Similar  guidelines  should  be  developed  for  all  levels  of  the  USCG,  providing 
assistance  in  their  establishment  and  conduct  of  an  effective  training  process,  as  appropriate. 

The  use  of  formal  training  centers,  located  on  or  near  a  duty  station,  should  be  considered.  A 
training  center  at  the  duty  station,  similar  to  those  at  shoreside  support  units,  would  be 
appropriate  for  larger  units,  including  shoreside  and  shipboard.  These  may  also  be  helpful  to 
smaller  units  when  located  nearby,  since  some  smaller  units  may  have  difficulty  in  making 
appropriate  training  spaces  available  on  a  regular  basis. 

A  detailed  transition  plan  should  be  developed  to  guide  the  shift  from  resident  to  nonresident 
training,  if  large-scale  nonresident  training  is  to  be  implemented.  The  initial  phase  of  the  plan 
should  generate  detailed  information  for  the  alternative  approaches  to  the  many  relevant  issues, 
and  decisions  made.  Strawmen  policies  and  procedures  should  be  developed  and  field-tested. 
The  initial  courses  implemented  should  be  evaluated,  focusing  on  both  the  course  and  training 
process,  to  identify  and  mitigate  problems.  The  marketing  of  nonresident  training  to  all  USCG 
personnel  should  be  an  important  part  of  this  initial  phase.  This  process  should  work  toward  the 
detailed  plan  and  schedule  for  later  large-scale  course  conversion  and  implementation.  The  later 
phases  should  accelerate  the  course  conversions,  to  take  advantage  of  the  developing 
infrastructure  and  readiness  for  nonresident  training.  The  goals  of  the  implementation  plan 
should  be  specification  of  a  capable  training  organization,  with  an  effective  training  process  at 
duty  stations,  and  the  appropriate  courses  converted  and  implemented  in  nonresident  training. 
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APPENDIX  A 


HANDS-ON  PROBLEMS 
DESCRIPTION  AND  SCORING 

This  appendix  provides  information  on: 

A.l  Hands-on  problems 

A.2  Scoring  of  hands-on  problems 

A.l  HANDS-ON  PROBLEMS 

The  5  hands-on  problems  are  listed  below,  including:  1)  the  symptoms  and  situation  information 
given  to  the  student;  2)  the  fault  inserted  into  the  AN/WSC-3  UHF  transceiver;  and  3)  the  action 
the  student  had  to  take  to  correct  the  malfunction.  The  student  was  given  only  the  symptoms  and 
situation  information. 

•  Problem  #1. 

-  Symptoms  and  Situation:  A  fellow  technician  was  running  BITE  tests  on  the 
AN/WSC-3  and  had  a  bunch  of  BITE’s  fail.  You  were  called  in  to  troubleshoot  the 
radio. 

-  Fault:  1 A1 A6  Translator  Bypass  Module  bad. 

-  Corrective  Action:  Replace  the  1 A1 A6  Translator  Bypass  Module. 

•  Problem  #2. 

-  Symptoms  and  Situation:  The  TCI  notifies  you  that  the  AN/WSC-3  will  not 
transmit  or  receive.  You  need  to  repair  the  radio  before  the  cutter  can  get  underway. 

-  Fault:  1A1A23  5  MHz  Oscillator  signal  bad. 

-  Corrective  Action:  Replace  the  1 A1A23  5  MHz  Oscillator. 

•  Problem  #3. 

—  Symptoms  &  Situation:  The  CO  tells  you  she  can  barely  hear  in  both  AM  and  FM 
on  the  AN/WSC-3.  Your  liberty  is  on  hold  until  you  repair  the  radio. 

-  Fault:  1A1A16  IF  Amplifier  Module  bad. 

-  Corrective  Action:  Replace  the  1 A1A16  IF  Amplifier  Module. 

•  Problem  #4. 

-  Symptoms  &  Situation:  Chief  calls  you  in  because  the  AN/WSC-3  does  not 
transmit.  Fix  it. 
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-  Fault:  Carrier  signal  from  1 A1 A8  Synthesizer  module  missing. 

-  Corrective  Action:  Replace  the  1A1A8  Synthesizer  module. 

•  Problem  #5. 

-  Symptoms  &  Situation:  The  ET1  installed  a  new  ANAVSC-3  and  it  doesn’t  work. 
He  wants  you  to  troubleshoot  and  repair  the  radio. 

-  Fault:  Jumpers  are  not  installed  on  1A1TB4. 

-  Corrective  Action:  Install  the  jumpers  correctly  on  1A1TB4. 

A.2  SCORING  OF  HANDS-ON  PROBLEMS 


Assessment  Structure.  An  assessment  structure  was  developed  to  identify  performance  criteria 
related  to  ANAVSC-3  troubleshooting  and  maintenance,  to  be  used  as  the  basis  forjudging 
student  proficiency  during  each  hands-on  problem.  A  single  assessment  structure  was  developed 
for  application  to  all  of  the  five  hands-on  problems.  The  structure’s  framework  focused  on  six 
dimensions  of  performance,  which  mirrored  the  general  troubleshooting  and  maintenance  process 
students  were  expected  follow  during  each  problem: 

•  Symptoms 

•  Sectionalization 

•  Localization 

•  Isolation 

•  Corrective  Action 

•  Overall  Performance  (Documentation  and  Safety) 

Several  task-items  were  identified  under  each  dimension,  also  generically  pertaining  to  all  5 
problems,  resulting  in  a  minimum  of  22  individual  task-items  and  overall  characteristics  that 
were  evaluated  for  each  hands-on  problem.  The  two  evaluators  independently  judged  student 
performance  for  each  task-item,  and  awarded  a  number  of  points  for  each.  Detailed  scoring 
criteria  were  developed  for  each  task-item,  to  provide  a  highly  structured  framework  for  the 
evaluator  judgments  and  awarding  of  points.  The  criteria  specified  up  to  5  levels  of  performance 
for  each  task-item  (0  through  4  points),  with  general  criteria  and  specific  examples  given  for  each 
point  level.  An  evaluation  criteria  guide  was  developed,  providing  detailed  information  to  guide 
the  evaluation  process.  An  accompanying  evaluation  form  was  developed  for  recording  of  the 
task-item  scores.  This  form  also  contained  abbreviated  guidance  information  for  the  judgment 
and  awarding  of  points  under  each  task-item. 

Student  Problem  Score.  Student  performance  solving  the  hands-on  problems  was 
independently  assessed  by  each  evaluator,  for  each  student.  This  resulted  in  two  sets  of  problem 
scores  for  each  student,  one  set  from  each  evaluator.  Scores  in  each  set  were  summed  (i.e., 
across  the  five  problems  for  each  evaluator)  to  yield  a  single  evaluator  score  for  each  student. 

The  two  evaluator  scores  were  averaged  to  yield  the  single  hands-on  problem  score  for  each 
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student  (i.e.,  student  problem  score).  The  scores  were  also  normalized  to  a  basis  of  100%  during 
this  process. 

The  student  problem  score  was  derived,  for  each  student,  on  the  basis  of: 

Task-item  score  =  Judgment  of  evaluator,  in  accordance  with  structured  scoring  criteria, 
weighted  according  to  task  importance. 

Dimension  score  =  Sum  of  task-item  scores  (multiple  task-items  for  each  dimension) 
Problem  score  =  Sum  of  dimension  scores  (across  6  dimensions) 

Evaluator  score  =  Sum  of  problem  scores  (across  5  problems) 

Student  problem  score  =  Mean  of  evaluator  scores  (across  2  evaluators) 

The  task-item  scores,  and  the  resulting  dimension  scores,  were  also  differentially  weighted  in 
accordance  with  their  relative  importance.  The  determination  of  task-item  and  dimension 
weights  was  accomplished  together  with  the  TRACEN  Petaluma  SMEs. 

Scoring  Weights.  The  evaluator  score  for  each  problem  was  arrived  at  by  summing  the 
individual  task-item  scores  assigned  by  the  evaluator,  with  transformation  in  accordance  with 
assigned  item  and  dimension  weights.  The  individual  task-item  assessments,  transformed  in 
accordance  with  assigned  weights,  were  summed  to  yield  the  respective  dimension  scores  for 
each  problem,  for  that  evaluator.  The  dimension  scores  were  also  transformed  in  accordance 
with  assigned  weights,  and  summed  to  yield  the  single  problem  score.  The  problems  were 
treated  equally,  with  their  scores  summed  to  yield  an  aggregate  hands-on  problem  score.  This 
aggregate  score  was  averaged  across  both  evaluators  to  generate  the  single  hands-on  problem 
score  for  the  student,  which  was  evaluated  to  determine  training  effectiveness. 

The  task-item  weights  and  the  dimension  weights  were  developed  with  the  assistance  of  the 
ANAVSC-3  SMEs  at  TRACEN  Petaluma.  Table  A-l  presents  the  weights  assigned  to  the 
individual  task  items,  with  the  task-items  grouped  under  the  respective  dimensions.  Table  A-2 
presents  the  dimension  weights. 


TABLE  A-l.  TASK  ITEM  WEIGHTS 


DIMENSIONS 

TASK  ITEMS 

fVHnnM 

SYMPTOMS 

Performance  Test 

2.0 

List  Faulty  Symptoms 

2.0 

List  Faulty  Equipment 

1.4 

Next  Troubleshooting  Procedure 

1.8 

SECTION  ALIZATION 

Additional  Tests  Needed 

1.8 

Identify  Most  Probable  Faulty  Functional 
Area(s) 

2.0 

LOCALIZATION 

BITE  Test 

1 

Identify  Most  Probable  Faulty  Module 

ISOLATION 

Explain  Method  of  Troubleshooting 

1.7 

Explain  Next  Troubleshooting  Point 

1.5 

Testing  -  Setup  and  Performance 

2.0 

Interpret  Performance 

2.0 

CORRECTIVE  ACTION 

Replace  Faulty  Module  or  Item 

1.5 

Alignment 

1.4 

Performance  Test 

OVERALL 

Documentation  Usage 

1.2 

PERFORMANCE 

Safety 

1.0 

The  item  weights  provided  the  relative  contribution  of  each  task  item  to  the  performance  under 
each  dimension.  For  example,  under  the  Localization  dimension,  the  BITE  Test  and  the  Identify 
Most  Probable  Faulty  Module  items  each  contributed  50%  of  the  performance  to  that  dimension; 
they  were  weighted  the  same  within  that  dimension.  Similarly,  the  dimension  weights  provided 
the  relative  contribution  of  each  dimension  to  the  performance  on  each  problem.  This  resulted  in 
a  two-tiered  weighting  system  for  scoring  student  performance 
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TABLE  A-2.  DIMENSION  WEIGHTS 


DIMENSIONS 

Dimension 

Weight 

SYMPTOMS 

1.8 

SECTIONALIZATION 

1.1 

LOCALIZATION 

2.7 

ISOLATION 

2.7 

CORRECTIVE  ACTION 

0.7 

OVER  ATT.  PERFORMANCE  (Documentation  & 

1.0 

Safety) 

Total  Dimension  Weight  10.0 


A-5 


[Blank] 


A-6 


APPENDIX  B 


STUDENT  CRITIQUE  QUESTIONNAIRES 


This  appendix  contains  the  three  versions  of  the  Student  C-School  Critique,  for  the  three  student 
groups: 


Duty  Station  Group  (CGSCH-5-DS) 
Remote  Group  (CGSCH-5-RM) 
Resident  Group  (CGSCH-5-RS) 
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Department  of  Transportation  DUTY  STATION  GROUP 

U.  S.  Coast  Guard 
CGCSCH-5-DS  (REV  7/96) 

C-SCHOOL  CRITIQUE  SHEET 


NAME:  _ 

Course:  _  Class  Number: _ 

In  order  to  improve  the  course  and  to  provide  a  measure  of  quality  assurance,  we  need  your 
feedback  on  this  critique.  Circle  the  number  next  to  each  statement  that  best  expresses  your 
opinion.  Space  is  provided  for  writing  amplifying  comments.  We  especially  encourage 
comments  an  any  statements  you  disagree  with.  Use  the  backside  of  pages  if  necessary. 

O-N/A  1-Strongly  Disagree  2-Disagree  3-Agree  4-Strongly  Agree 

1 .  Iam  very  confident  of  my  ability  to  maintain  this  equipment.  0  1  2 

2.  Testing  clearly  measured  my  ability  to  perform  each  objective.  0  1  2 


3.  Plenty  of  practice  is  provided  for  each  objective.  0  1  2 


4.  Proper  troubleshooting  techniques  are  taught,  and  tested.  0  1  2 


5.  Troubleshooting  problems  are  realistic. 


0  1  2 


6.  Course  is  presented  in  a  logical  sequence. 


7.  Course  length  is  ideal. 


8.  Proper  tools  and  test  equipment  are  always  available  in  the 
computer  media. 

9.  Spare  parts  are  available  and  ready  for  use  in  computer  media. 


1 0.  Safety  equipment  is  always  available. 


0  1  2  3.4 

0  12  3  4 

0  12  3  4 

0  12  3  4 

0  12  3  4 


1 1 .  Training  and  visual  aids  enhanced  learning. 


0  12  3  4 


12  Technical  manuals  are  current,  legible,  and  accessible. 


0  1  2 


13.  HELP  information  on  the  computer  was  understandable  and  0  1 

enhanced  training. 

14.  Course  work  directly  relates  to  course  objectives.  0  1 
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1 5.  Welcome  aboard  package  is  very  helpful. 


16.  The  facilitator  clearly  and  concisely  answered  all  questions. 


17.  Facilitator  maintained  control. 


1 8.  Facilitator  is  knowledgeable,  skillful,  and  confident  in  assisting 
training  process. 


1 9.  Facilitator  strictly  enforced  safety  standards. 


20.  Training  environment  was  comfortable  and  functional. 


2 1 .  There  is  good  visibility  in  training  room/spaces. 


22.  There  were  no  distractions  during  training. 


23.  Training  time  was  never  wasted. 


0  12  3  4 

0  12  3  4 

0  12  3  4 

0  12  3  4 

0  12  3  4 

0  12  3  4 

0  12  3  4 

0  12  3  4 

0  12  3  4 


B-4 


24.  I  was  well  prepared  to  take  this  AN/WSC-3  course. 


25.  The  AN/WSC-3  course  was  directly  relevant  to  my  job. 


26.  The  quality  of  training  was  satisfactory. 


27.  The  course  was  effective  overall. 


28.  The  AN/WSC-3  computer-based  course  was  as  effective  as 
traditional  classroom/lab  course. 


29.  The  level  of  AN/WSC-3  instruction  detail  was  sufficient. 


30.  The  AN/WSC-3  computer-based  course  was  challenging. 


31.  The  amount  of  general  theory  provided  was  sufficient. 


32.  The  computer  medium  was  satisfactory  for  the  practice 
problems. 


0  1  2 


3  4 


33.  Some  AN/WSC-3  equipment  hands-on  training  should  be 
provided  after  completing  the  computer-based  part  of  the 
course. 

34.  The  computer-based  training  materials  were  user  friendly  and  0  12  3  4 

easy  to  use. 

35.  The  computer  and  software  worked  satisfactorily.  0  12  3  4 


36.  I  had  no  difficulty  learning  to  use  the  computer  for  this  course.  0  12  3  4 

37.  The  training  environment  (room,  table/desk  where  trained  on  0  12  3  4 

the  computer)  was  satisfactory. 

38.  The  time  made  available  for  me  to  do  the  course  was  adequate.  0  12  3  4 

39.  Computer-based  training  at  duty  stations  can  be  effective  in  the  0  12  3  4 

Coast  Guard. 

40.  My  command  strongly  supported  this  AN/WSC-3  computer-  0  12  3  4 

based  training. 

41.  I  had  no  difficulty  in  setting  up  and  conducting  the  computer-  0  12  3  4 

based  training. 

43.  There  was  very  little  interference  from  others  during  my  0  12  3  4 

training  sessions. 
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0  12  3  4 


44.  The  computer  and  all  necessary  guidance  and  training  materials 
were  satisfactorily  received  at  my  duty  station. 


Please  provide  your  opinions  to  the  following  questions.  Use  page  backsides  or  additional  sheets 
if  necessary. 

45.  The  most  beneficial  part  of  this  course  was _ _ _ _ 


46.  The  least  beneficial  part  of  this  course  was _ _ _ 

47.  What  additional  information,  materials  or  help  would  have  improved  your  training 

experience? _ _ _ 


48.  What  additional  AN/WSC-3  topics  would  you  like  to  see  added  to  this  course? _ 

49.  In  what  other  Coast  Guard  training  areas  or  courses  might  computer-based  training  be 

effective? _ _ _ _ 


50.  My  suggestions  for  improving  this  type  of  individual  computer-based  training  at  duty 
stations  are: _ _ _ 


Please  use  the  space  below  to  provide  us  any  remarks  (positive  or  negative)  concerning  the 
course.  Use  page  backsides  or  additional  paper  if  needed.  Thank  you. 
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Department  of  Transportation 
U.  S.  Coast  Guard 
CGCSCH-5-RM  (REV  7/96) 


REMOTE  GROUP 


C-SCHOOL  CRITIQUE  SHEET 


NAME:  (optional)  _ 

Course:  _  Class  Number: _ 

In  order  to  improve  the  course  and  to  provide  a  measure  of  quality  assurance,  we  need  your 
feedback  on  this  critique.  Circle  the  number  next  to  each  statement  that  best  expresses  your 
opinion.  Space  is  provided  for  writing  amplifying  comments.  We  especially  encourage 
comments  an  any  statements  you  disagree  with.  Use  the  backside  of  pages  if  necessary. 

O-N/A  1-Strongly  Disagree  2-Disagree  3-Agree  4-Strongly  Agree 

1 .  Iam  very  confident  of  my  ability  to  maintain  this  equipment.  0  12  3  4 

2.  Testing  clearly  measured  my  ability  to  perform  each  objective.  0  12  3  4 

3.  Plenty  of  practice  is  provided  for  each  objective.  0  12  3  4 


4.  Proper  troubleshooting  techniques  are  taught,  and  tested. 


0  12  3  4 


5.  Troubleshooting  problems  are  realistic. 


0  12  3  4 


6.  Course  is  presented  in  a  logical  sequence. 


0  12  3  4 
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7. 


Course  length  is  ideal 


8.  Proper  tools  and  test  equipment  are  always  available  in  the 
computer  media. 

9.  Spare  parts  are  available  and  ready  for  use  in  computer  media. 

10.  Safety  equipment  is  always  available. 


1 1.  Training  and  visual  aids  enhanced  learning. 


12  Technical  manuals  are  current,  legible,  and  accessible. 


13.  HELP  information  on  computer  is  understandable  and 
enhances  training. 

14.  Course  work  directly  relates  to  course  objectives. 


15.  Welcome  aboard  package  is  very  helpful. 


16.  The  facilitator  clearly  and  concisely  answered  all  questions. 


17.  Facilitator  maintained  control. 


0  1  2 


3  4 


18.  Facilitator  is  knowledgeable,  skillful,  and  confident  in  assisting  0  12  3  4 

training  process. 

19.  Facilitator  strictly  enforced  safety  standards.  0  12  3  4 

20.  Training  environment  was  comfortable  and  functional.  0  12  3  4 

21.  There  is  good  visibility  in  training  room/spaces.  0  12  3  4 

22.  There  were  no  distractions  during  training.  0  12  3  4 

23.  Training  time  was  never  wasted.  0  12  3  4 

24.  I  was  well  prepared  to  take  this  AN/WSC-3  course.  0  12  3  4 

25.  The  AN/WSC-3  course  was  directly  relevant  to  my  job.  0  12  3  4 


26.  The  quality  of  training  was  satisfactory. 


0  1  2 


3  4 


27.  The  course  was  effective  overall. 


0  1  2 


3  4 


28.  The  AN/WSC-3  computer-based  course  was  as  effective  as  a 
traditional  classroom/lab  course. 

29.  The  level  of  AN/WSC-3  instruction  detail  was  sufficient. 


0  12  3  4 


0  12  3  4 


30.  The  AN/WSC-3  computer-based  course  was  challenging. 


0  12  3  4 


3 1 .  The  amount  of  general  theory  provided  was  sufficient. 


0  12  3  4 


32.  The  computer  medium  was  satisfactory  for  the  practice 
problems. 


0  12  3  4 


33.  Some  AN/WSC-3  equipment  hands-on  training  should  be 
provided  after  completing  the  computer-based  part  of  the  course. 

34.  The  computer-based  training  materials  were  user  friendly  and 
easy  to  use. 


0  12  3  4 


0  12  3  4 


35.  The  computer  and  software  worked  satisfactorily. 


0  12  3  4 


36.  I  had  no  difficulty  learning  to  use  the  computer  for  this  course.  0  12  3  4 
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0  1  2 


3  4 


37.  The  training  environment  (room,  table/desk  where  trained  on 
the  computer)  was  satisfactory. 

38.  The  time  made  available  for  me  to  do  the  course  was  adequate.  0  12  3  4 

39.  Computer-based  training  at  duty  stations  can  be  effective  in  the  0  12  3  4 

Coast  Guard. 

Please  provide  your  opinions  to  the  following  questions.  Use  page  backsides  or  additional  sheets 
if  necessary. 

40.  The  most  beneficial  part  of  this  course  was _ 

41 .  The  least  beneficial  part  of  this  course  was _ 

42.  What  additional  information,  materials  or  help  would  have  improved  your  training 

experience? _ _ 

43.  What  additional  ANAVSC-3  topics  would  you  like  to  see  added  to  this  course? _ 

44.  In  what  other  Coast  Guard  training  areas  or  courses  might  computer-based  training  be 

effective? _ 
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45.  My  suggestions  for  improving  this  type  of  individual  computer-based  training  at  duty 
stations  are: - - - - 


Please  use  the  space  below  to  provide  us  any  remarks  (positive  or  negative)  concerning  the 
course.  Use  page  backsides  or  additional  paper  if  needed.  Thank  you. 


# 
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Department  of  Transportation 
U.  S.  Coast  Guard 
CGCSCH-5-RS  (REV  7/96) 


RESIDENT  GROUP 


C-SCHOOL  CRITIQUE  SHEET 

NAME:  (optional)  _ 

Course:  _  Class  Number: _ 

In  order  to  improve  the  course  and  to  provide  a  measure  of  quality  assurance,  we  need  your 
feedback  on  this  critique.  Circle  the  number  next  to  each  statement  that  best  expresses  your 
opinion.  Space  is  provided  for  writing  amplifying  comments.  We  especially  encourage 
comments  an  any  statements  you  disagree  with.  Use  the  backside  of  pages  if  necessary. 

O-N/A  1-Strongly  Disagree  2-Disagree  3-Agree  4-Strongly  Agree 

1 .  Iam  very  confident  of  my  ability  to  maintain  this  equipment.  0  12  3  4 

2.  Testing  clearly  measured  my  ability  to  perform  each  objective.  0  12  3  4 

3.  Plenty  of  practice  is  provided  for  each  objective.  0  12  3  4 


4.  Proper  troubleshooting  techniques  are  taught,  and  tested.  0  12  3  4 


5.  Troubleshooting  problems  are  realistic.  0  12  3  4 


0  12  3  4 


0  12  3  4 


6.  Course  is  presented  in  a  logical  sequence. 


8.  Proper  tools  and  calibrated  test  equipment  are  always  available. 
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9.  Spare  parts  are  plentiful  and  ready  for  issue. 


10.  Safety  equipment  is  always  available. 


1 1 .  Training  and  visual  aids  enhanced  learning. 


12  Technical  manuals  are  current,  legible,  and  accessible. 


13.  Handouts  are  legible  and  enhance  training. 


14.  Course  work  directly  relates  to  course  objectives. 


15.  Welcome  aboard  package  is  very  helpful. 


1 6.  The  instructor  clearly  and  concisely  answered  all  questions. 


17.  Instructor  maintained  interest  and  control  of  class. 


18.  Instructor  is  knowledgeable,  skillful,  and  confident. 


19.  Instructor  strictly  enforced  safety  standards. 


20.  Classroom  and  labs  are  comfortable,  and  functional. 


21.  There  is  good  visibility  in  classroom  and  lab. 


22.  There  are  no  distractions  to  the  class. 


23.  Class  time  is  never  wasted. 


24.  I  was  well  prepared  to  take  this  AN/WSC-3  course. 


25.  The  AN/WSC-3  course  was  directly  relevant  to  my  job. 


26.  The  quality  of  training  was  satisfactory. 


27.  The  course  was  effective  overall. 


28.  The  level  of  AN/WSC-3  instruction  detail  was  sufficient. 


0  12  3  4 


29.  The  amount  of  general  theory  provided  was  sufficient. 


0  12  3  4 


Please  provide  your  opinions  to  the  following  questions.  Use  page  backsides  or  additional  sheets 
if  necessary. 

30.  The  most  beneficial  part  of  this  course  was _ _ _ 


3 1 .  The  least  beneficial  part  of  this  course  was _ 

32.  What  additional  information,  materials  or  help  would  have  improved  your  training 

experience? _ _ _ 


33.  What  additional  AN/WSC-3  topics  would  you  like  to  see  added  to  this  course?. 


Please  provide  us  any  remarks  (positive  or  negative)  concerning  the  course.  Use  page  backsides 
or  additional  paper  if  needed.  Thank  you. 
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APPENDIX  C 


EXPERIMENT  MATERIALS 

This  appendix  contains  summary  information  about  experiment  materials,  presented  in  the 
following  subsections: 

C.l  Training  Materials 

C.2  CBT  Duty  Station  Group 

C.3  CBT  Remote  Group 

C.4  Resident  Group 

C.5  TRACEN  Petaluma  Evaluation 

C.1  TRAINING  MATERIALS 

The  three  student  groups  used  the  same  reference  publications  during  their  respective  courses, 
which  were: 


ANAVSC-3(v)7  Technical  Manuals 

AN/WSC-3(v)7  Student  Guide 

Student  Corrective  Maintenance  Log  Sheets 

The  two  volume  set  of  AN/WSC-3  technical  manuals,  which  were  developed  and  supplied  by  the 
U.S.  Navy,  are  the  standard  reference  for  operating,  troubleshooting,  and  maintaining  the  UHF 
transceivers.  These  manuals  were  augmented  by  the  AN/WSC-3(v)7  UHF  transceiver  Student 
Guide  (course  book)  developed  by  TRACEN  Petaluma.  The  course  book  provided  additional 
guidance  and  training-related  information.  The  Student  Corrective  Maintenance  Log  Sheets  were 
blank  forms  used  by  students  to  write  down  observations  and  other  information  during  the 
troubleshooting  and  maintenance  work.  Students  were  instructed,  in  each  group,  to  use  these 
forms  during  the  troubleshooting  problems  (These  forms  were  also  used  by  the  students  during 
the  hands-on  testing  problems).  These  three  documents  were  identical  for  each  of  the  three 
groups. 

The  CBT  Duty  Station  (DS)  and  CBT  Remote  (RM)  groups  used  the  AN/WSC-3  CBT  course 
medium,  in  addition  to  the  technical  manuals  and  course  book.  The  computer  medium  contained 
the  bulk  of  course  material,  which  was  stored  on  the  laptop  PC’s  hard  drive.  In  addition,  the 
CBT  students  received  a  3-page  document,  Computer  Tum-On/Off  Procedures,  to  guide  them  in 
the  mechanics  of  operating  the  laptop  computer. 

The  Resident  (RS)  group  used  the  equipment  and  other  media  in  the  AN/WSC-3  lab  at  TRACEN 
Petaluma  (e.g.,  AN/WSC-3  transceivers)  during  their  course. 

Additional  information  about  the  two  course  versions  is  provided  in  Section  2.1.2,  AN/WSC-3 
Course. 


C-l 


C.2  CBT  DUTY  STATION  GROUP 


Materials  were  developed  for,  and  sent  to,  three  persons  at  each  duty  station:  1)  Commanding 
Officer;  2)  Facilitator  (The  person,  usually  the  student’s  immediate  supervisor,  who  would 
oversee  the  AN/WSC-3  CBT  course  conduct);  and  3)  Student.  Introductory  letters  were  sent  to 
all  three  participants,  a  month  prior  to  start  of  the  duty  station  training  period.  The  set  of 
materials  sent  to  each  duty  station  consisted  of  the  following,  each  of  which  is  explained  briefly 
below: 


Facilitator  Guideline 
Facilitator  Job  Aids 
Student  Guideline 

Duty  Station  Facilitator  Questionnaire 
Student  Background  Questionnaire 
Pretest 

Facilitator  Guideline.  This  document  was  used  by  the  facilitator  to  oversee  the  CBT-related 
activities  at  the  duty  station.  It  contained  a  series  of  tasks  and  subtasks  to  be  accomplished  by  the 
Facilitator,  independently  and  with  the  student.  It  also  provided  guidance  about  the  tasks  and 
subtasks,  facilitator  job  aids,  and  a  detailed  schedule  for  the  activities. 

Facilitator  Job  Aids.  Several  job  aids  were  developed  to  assist  the  Facilitator  in  performing 
required  tasks.  They  included: 

-  Facilitator  Check-Off  Sheet  -  Listing  of  tasks,  subtasks,  completion  dates,  guideline 
reference  pages,  and  spaces  for  recording  task  completions. 

-  Facilitator  Tasks  Outline  -  Flowchart  of  duty  station  activities. 

-  Packing  List  -  List  of  received  computer  and  material. 

-  Task/Subtask  Schedule  -  Detailed  schedule  of  data  collection  and  training  activities. 

-  HELP  Process  -  Summary  of  HELP  available  and  procedures. 

-  Return  Shipping  Guide  -  Summary  of  return  shipment  package  contents  (2 
packages) 

Student  Guideline.  The  1-page  document  provided  general  guidance  to  the  student,  addressing 
training  session  management,  use  of  materials  and  obtaining  help. 

Duty  Station  Facilitator  Questionnaire.  This  22-item  questionnaire  requested  judgments  by 
the  Facilitator  about  the  CBT  course.  It  was  scheduled  to  be  completed  after  the  student  finished 
the  course. 

Student  Background  Questionnaire.  This  questionnaire  collected  detailed  background 
information  about  each  student.  The  information  categories  included: 
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Prior  education  (Civilian  -  High  school,  trade  school,  college,  other) 

Prior  education  (Military  -  A-School,  B-School,  C-Schools,  other) 

CLEP/DANTES 
Certifications  (e.g.,  FCC) 

Service  time  and  assignment  history  (Unit,  type,  arrival/departure  dates,  job  title) 

History  of  equipment  experience  (In  addition  to  formal  training) 

AN/WSC-3  experience  (Operations,  Planned  Maintenance  System,  troubleshooting) 

Pretest.  The  Pretest  was  administered  by  the  Facilitator  prior  to  the  start  of  the  training.  See 
additional  description  under  the  subsection  Section  2.1.5,  Performance  Measurement  Tools. 

C.3  CBT  REMOTE  GROUP 

The  RM  group  used  the  same  materials  as  the  DS  group,  excepting  they  had  modifications 
necessary  to  conform  with  administration  of  training  in  the  school  environment  at  TRACEN 
Petaluma.  Introductory  letters  were  given  to  the  Facilitator  (at  TRACEN  Petaluma)  and  sent  to 
each  student  prior  to  reporting  for  the  course,  which  was  conducted  at  TRACEN  Petaluma.  The 
TRACEN  Petaluma  facilitator  performed  the  same  role  as  the  facilitators  at  each  of  the  duty 
stations.  The  materials  used  by  the  RM  group  included  (See  the  above  Section  C.2,  Duty  Station 
Group,  for  an  explanation  of  each): 

Facilitator  Guideline 
Facilitator  Job  Aids 
Student  Guideline 

Duty  Station  Facilitator  Questionnaire 
Student  Background  Questionnaire 
Pretest 

C.4  RESIDENT  GROUP 

The  RS  group  materials  used  prior-to,  and  during,  the  training  differed  from  those  of  the  other 
two  groups.  An  introductory  letter  was  sent  to  the  RS  group  students,  similar  to  that  of  the  other 
groups.  The  other  materials  consisted  of: 

Student  Background  Questionnaire 
Pretest 

Instructor  Guide  supplement 

The  Student  Background  Questionnaire  and  Pretest  administered  to  these  students  was  identical 
to  that  of  the  other  groups. 

Instructor  Guide  Supplement.  The  standard  AN/WSC-3  Course’s  Instructor  Guide  was 
supplemented  by  material  addressing  the  experiment,  similar  to  that  in  the  Facilitator  Guide  for 
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the  other  two  groups.  This  included,  for  example,  a  Training  Project  Introduction,  Instructor 
Outline,  which  was  used  to  brief  students  at  the  start  of  the  course.  With  the  exception  of  the 
initial  data  collection  (i.e..  Student  Background  Questionnaire  and  Pretest )  the  resident  course 
was  conducted  in  the  traditional  fashion,  and  required  no  experiment-unique  tasks  by  the 
instructor. 

C.5  TRACEN  PETALUMA  EVALUATION 

The  post-training  and  testing  conducted  at  TRACEN  Petaluma  was  performed  the  same  for  all 
groups,  and  used  the  same  materials,  as  follows: 

Faulted  AN/WSC-3  modules 
Evaluator  Guideline 

Scoring  Criteria  Guideline;  Scoring  sheet 
AN/WSC-3  transceiver  and  lab  materials 

Faulted  AN/WSC-3  Modules.  The  5  hands-on  problems  were  effected  by  inserting  faulted 
modules  into  the  AN/WSC-3  transceivers  in  the  lab  at  TRACEN  Petaluma.  The  faulted  modules 
were  developed  by  the  TRACEN  Petaluma  SMEs. 

Evaluator  Guideline.  This  document  was  used  by  the  Evaluators  to  oversee  the  testing 
activities.  It  contained  a  series  of  tasks  and  subtasks  to  be  accomplished  by  the  Evaluators.  It 
also  provided  guidance  about  the  tasks  and  subtasks,  evaluator  job  aids,  and  a  detailed  schedule 
of  testing  activities. 

Scoring  Criteria  Guide,  and  Scoring  Sheet.  The  guide  provided  direction  to  the  evaluators  for 
judging  aspects  of  student  performance  during  the  hands-on  problems.  It  identified  problem 
dimensions,  task-items  and  scoring  criteria,  with  examples.  See  Section  2.1.5,  Performance 
Measurement  Tools,  for  additional  information. 

The  Scoring  Sheet  was  a  data  recording  job  aid  used  by  the  evaluators  to  record  their  judgment 
scores  during  the  hands-on  problems.  It  also  provided  summary  guidance  to  the  evaluators,  as  an 
accompaniment  to  the  Scoring  Criteria  Guide.  See  Section  2.1.5,  Performance  Measurement 
Tools  subsection  for  additional  information. 

AN/WSC-3  Transceiver  and  Lab  Materials.  The  hands-on  testing  was  conducted  in  a  lab  at 
TRACEN  Petaluma,  containing  6  rack-mounted  AN/WSC-3  transceivers.  Other  operational 
equipment  was  also  rack-mounted  nearby,  appropriately  connected  to  the  AN/WSC-3 
transceiver.  A  separate  set  of  test  equipment  (e.g.,  oscilloscope)  was  in  an  adjoining  rack, 
dedicated  for  use  with  that  AN/WSC-3  unit. 

Each  student  received  a  set  of  materials  at  the  start  of  each  hands-on  testing  problem.  It  included 
the  AN/WSC-3  Technical  Manuals  (Volumes  1  and  2),  and  tools  (e.g.,  test  pin  adapters).  All  the 
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materials  and  equipment  necessary  for  troubleshooting  the  AN/WSC-3  UHF  transceiver  were 
provided  at  each  AN/WSC-3  station  at  the  start  of  each  problem. 

Appendix  A,  Hands-On  Problems,  Description  and  Scoring,  provides  additional  information  on 
the  scoring  process  used  to  evaluate  student  performance  during  the  hands-on  problems. 
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APPENDIX  D 


EXPERIMENT  PROCEDURES 

Details  of  the  experiment  procedures  are  summarized  in  this  appendix.  They  are  presented  in 
sections  under  each  student  group,  as  follows: 

D.  1  CBT  Duty  Station  Procedures 

D.2  CBT  Remote  Procedures 

D.3  Resident  Procedures 

D.l  CBT  DUTY  STATION  PROCEDURES 

Initial  contact  with  each  duty  station  was  made  approximately  one-month  before  start  of  that  duty 
station’s  experiment-support  and  training  activities.  The  Commanding  Officer  was  contacted 
initially,  with  subsequent  facilitator  contact.  The  project  was  introduced,  with  cooperation 
requested. 

The  set  of  experiment  and  training  materials  were  shipped  to  the  duty  station  sufficiently  in 
advance,  to  allow  for  a  one-week  preparation  period  by  the  facilitator,  prior  to  the  start  of  the 
CBT  training  period.  A  two-week  CBT  training  period  was  specified,  during  which  the 
AN/WSC-3  course  had  to  be  conducted.  The  facilitator  administered  the  Student  Background 
Questionnaire  and  Pretest  at  the  start  of  this  training  period.  The  facilitator  at  each  duty  station 
was  the  coordinator  for  the  experiment  support  activities  and  AN/WSC-3  training.  The 
facilitator  did  not  get  involved  in  the  student’s  training  process. 

Immediately  following  completion  of  the  two-week  training  period,  the  student  traveled  to 
TRACEN  Petaluma  for  the  post-training  testing.  The  testing  occupied  a  three  day  period  at 
TRACEN  Petaluma,  including  hands-on  training  as  part  of  the  CBT  course,  lab  familiarization, 
administration  of  the  Posttest,  administration  of  the  hands-on  test  problems,  and  administration 
of  the  C-School  Critique. 

Facilitator  Tasks.  The  facilitator  at  each  duty  station  was  the  coordinator  for  the  experiment 
support  activities  and  AN/WSC-3  training.  It  was  necessary  to  have  a  capable  facilitator  to 
achieve  experimental  control  across  the  DS  group  students.  For  this  reason,  extensive  guidance 
materials  and  job  aids  were  developed  for  the  facilitator,  and  tailored  to  the  specific  schedule  for 
that  DS  group  section.  The  schedule  of  facilitator  tasks,  which  are  representative  of  the  schedule 
of  CBT-related  activities  at  each  duty  station,  is  presented  in  Figure  D-l. 

The  complete  set  of  training  and  experiment  materials,  including  the  laptop  computer,  were 
shipped  to  the  facilitator  at  each  duty  station  to  arrive  more  than  a  week  before  the  scheduled 
start  of  the  CBT  training  period.  This  provided  the  facilitator  with  approximately  a  week  to 
prepare  for  the  training.  The  facilitator  was  the  project’s  representative  at  each  duty  station.  The 
facilitator’s  tasks  concerned: 
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Making  arrangements  with  the  student  and  other  affected  staff  for  conduct  of  the  training 
Receiving  and  storing  the  materials 

Administration  of  the  Student  Background  Questionnaire  and  Pretest 
Oversight  of  the  student’s  training  activities 
Completion  of  the  Facilitator  Questionnaire 
Return  of  the  materials 
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Figure  D-l.  DUTY  STATION  FACILITATOR  TASK/SUBTASK  SCHEDULE 
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Training  Period.  A  two- week  training  period  was  allowed  for  conduct  of  the  AN/WSC-3  CBT 
course  (Estimates  of  training  time  ranged  from  6  to  15  hours).  The  guidance  stated  that  the 
student  must  not  begin  training  before  the  start  of  the  two-week  period,  and  must  complete  the 
training  by  the  end  of  the  period.  The  training  period  commenced  on  a  Monday,  and  ended  on 
the  Friday  twelve  days  later.  Otherwise,  the  training  schedule  was  left  to  the  discretion  of  the 
duty  station.  Similarly,  general  guidance  was  provided  on  workspace  requirements  (e.g.,  quiet, 
lighting,  sufficient  desktop  space),  with  the  actual  arrangements  left  to  the  duty  station. 
Generally  speaking,  the  only  firm  requirement  placed  on  the  duty  station  was  the  two-week 
period  in  which  the  training  had  to  be  conducted. 

Travel  to  TRACEN  Petaluma.  Each  student  traveled  to  TRACEN  Petaluma  on  the  Sunday 
following  the  end  of  the  two-week  training  period  (i.e.,  14  days  after  the  start  of  the  training 
period). 

Hands-On  Training.  A  limited  amount  of  hands-on  training  with  an  actual  AN/WSC-3  UHF 
transceiver  was  considered  an  essential  part  of  the  AN/WSC-3  CBT  course.  This  training  was 
deemed  necessary  to  assist  the  transfer  of  skills  and  knowledge  learned  in  the  CBT  context  to 
that  of  the  actual  hardware.  A  two-hour  hands-on  training  session,  addressing  the  location  of 
AN/WSC-3  components  and  test  points,  and  planned  maintenance  procedures,  was  developed  to 
meet  this  requirement.  The  hands-on  training  for  all  affected  students  (DS  and  RM  groups)  was 
conducted  in  the  AN/WSC-3  lab  at  TRACEN  Petaluma,  prior  to  the  start  of  testing. 

The  outline  of  the  hands-on  training  for  the  DS  and  RM  groups  is  presented  in  Table  D-l. 

TABLE  D-l.  HANDS-ON  TRAINING  OUTLINE 


1 .  Familiarization  -  Instructor  Indoctrination 

•  Tools 

•  Equipment  configuration 

•  Test  Equipment 

2.  Run  performance  check 

3.  Perform  BITE  tests 

4.  Test  Point  Lab 

5.  Signal  Injection  Lab 

6.  Perform  PMS  Procedure  M-l 

7.  Module  Location  Lab 
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T .ahnratorv  Familiarization.  A  2-hour  laboratory  familiarization  session  was  developed  for  the 
DS  and  RM  group  students.  Its  purpose  was  to  acclimate  these  students  with  the  AN/WSC-3  lab 
(  and  its  general  environment,  since  lab  familiarity  could  effect  student  performance,  and  could  be 

confounded  with  the  hands-on  test  performance  between  the  CBT  and  RS  groups.  The  RS  group 
students  received  their  training  in  the  AN/WSC-3  lab,  over  a  5-day  period,  and  hence  had  a 
’  relatively  high  level  of  lab  familiarity  by  the  time  hands-on  testing  began.  Although  it  was  not 

possible  to  provide  an  equivalent  level  of  lab  familiarization  for  the  DS  and  RM  students  in  the 
small  amount  of  time  available,  a  2-hour  period  was  deemed  adequate. 

The  hands-on  training  and  lab  familiarization  contents  were  integrated  into  a  single  module,  for 
application  near  the  start  of  the  TRACEN  Petaluma  testing  activities.  Note,  the  RS  students  were 
allowed  an  equivalent  amount  of  unstructured  familiarization  time  in  the  lab  (2  hours),  following 
completion  of  training. 

TRACEN  Petaluma  Post-Training  Testing.  The  testing  procedures  that  were  followed  at 
TRACEN  Petaluma,  after  arrival  of  the  DS  group  students  Sunday,  were  as  follows  (See  the 
testing  schedule  in  Table  D-2): 

•  Testing  Introduction  -  Addressing  the  testing  schedule,  including  hands-on  training  and 
laboratory  familiarization. 

•  Hands-On  Training  and  Laboratory  Familiarization  -  Conducted  during  a  4-hour 
period  Monday  morning. 

•  Posttest  -  Administration  of  the  Posttest  Monday  afternoon  (Approximately  1  hour) 

•  Hands-On  Testing  -  First  subsection  of  three  students  began  testing  Tuesday  morning. 
Each  student  had  to  solve  three  separate  problems,  one  each  on  three  different  AN/WSC- 
3  transceivers.  The  second  subsection  began  testing  Tuesday  afternoon,  addressing  the 
same  three  problems.  Both  subsections  concluded  testing  Wednesday,  addressing  the 
final  two  problems  (again,  with  a  morning  and  an  afternoon  subsection  of  students). 

•  C-School  Critique  -  The  modified  C-School  Critique  was  individually  administered  to 
each  student  on  Wednesday,  immediately  following  the  student’s  completion  of  the  final 
hands-on  test  problem. 
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TABLE  D-2.  SCHEDULE  OF  CBT  DUTY  STATION  GROUP  TESTING  ACTIVITIES 


Time 

ebsh 

Monday 

ES0K5!^^H 

0730 

Introduction 
(1/2  hr.) 

Hands-on  Tests 
(AN/WSC-3  Lab) 

Hands-on  Tests 
(AN/WSC-3  Lab) 

0800 

Lab  Familiarization 
&  Hands-on 
training 

Problems  1  -3 
Subsection  A 

Problems  4,5 
Subsection  B 

0900 

1000 

1100 

C-School  Critique 

1200 

Hands-on  Tests 
(AN/WSC-3  Lab) 

Hands-on  Tests 
(AN/WSC-3  Lab) 

1300 

Post-test  (1  hour) 

Problems  1-3 
Subsection  B 

Problems  4,5 
Subsection  A 

1400 

m 

Students 

Arrive 

1600 

C-School  Critique 
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D.2  CBT  REMOTE  GROUP  PROCEDURES 


The  RM  group  students  reported  to  TRACEN  Petaluma  the  day  before  the  start  of  training  and 
experiment  activities,  similar  to  the  RS  students  (i.e.,  Sunday).  The  students  gathered  Monday 
morning  for  the  start  of  training,  in  a  classroom  remote  from  the  AN/WSC-3  lab  (where  the 
hands-on  testing  was  conducted).  The  facilitator  provided  an  introduction,  and  administered  the 
Student  Background  Questionnaire  and  Pretest  at  the  start  of  training,  Monday  morning.  The 
AN/WSC-3  CBT  course  then  commenced.  The  students  had  two-plus  days  in  which  to  complete 
the  course,  before  the  start  of  testing  late  Wednesday  morning.  The  students  were  permitted  to 
train  at  their  own  pace,  with  the  provision  that  they  finish  the  course  by  late  Wednesday  morning. 

The  post-training  and  testing  process  began  late  Wednesday  morning,  and  concluded  Friday 
afternoon.  This  testing  process  was  identical  to  that  of  the  DS  and  RS  groups,  with  the  exception 
of  the  time  schedule,  which  differed  for  each  group. 

Facilitator  Tasks.  The  RM  group  facilitator  had  no  special  preparation,  other  than  being  given 
guidance  materials  similar  to  those  provided  to  the  duty  station  facilitators,  about  one-week  prior 
to  the  start  of  training.  The  RM  facilitator  had  the  same  role  as  that  of  the  DS  facilitators  (i.e., 
coordinate  the  experiment  support  activities  and  AN/WSC-3  training,  and  not  get  involved  in  the 
student’s  training  process),  with  similar  tasks.  They  received  comparable  guidance  and  job  aid 
materials.  The  RM  facilitator’s  tasks  concerned: 

Administration  of  the  Student  Background  Questionnaire  and  Pretest 
Oversight  of  the  student’s  training  activities 
Completion  of  the  Facilitator  Questionnaire 

Hands-On  Training  and  Lab  Familiarization.  The  RM  students  received  the  same  integrated 
hands-on  training  and  lab  familiarization  as  that  conducted  for  the  DS  group,  in  accordance  with 
the  same  rationale  (See  Section  D.l,  Duty  Station  Procedures,  above). 

TRACEN  Petaluma  Procedures  (Pre-training,  training,  post-training  and  testing).  All  RM 
group  activities  were  conducted  at  TRACEN  Petaluma.  The  procedures  associated  with  the  pre¬ 
training,  AN/WSC-3  CBT  course,  post-training  and  testing  activities,  were  (See  the  training  and 
testing  schedule  in  Table  D-3): 

•  Introduction  -  Performed  by  the  facilitator  Monday  morning,  addressing  experiment  and 
training  activities. 

•  Student  Background  Questionnaire  -  Administered  by  the  facilitator,  Monday  morning. 

•  Pretest  -  Administered  by  the  facilitator,  Monday  morning. 

•  AN/WSC-3  CBT  Course  -  Started  Monday  morning  at  0930,  and  continued  until 
Wednesday  morning  at  1030.  The  facilitator  monitored  student  progress  to  assure 
completion  by  the  scheduled  time.  The  students  trained  at  their  own  pace,  and  were  able 
to  come-and-go,  with  the  provision  that  they  completed  the  course  on  schedule. 
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•  Testing  Introduction  -  Performed  by  the  evaluators  Wednesday  morning  at  1 1 00, 
addressing  the  testing  schedule,  including  hands-on  training  and  laboratory 
familiarization. 

•  Hands-On  Training  and  Laboratory  Familiarization  -  Conducted  during  a  4-hour 
period  Wednesday  afternoon. 

•  Posttest  -  Administration  of  the  Posttest  Wednesday  afternoon  (Approximately  1  hour). 

•  Hands-On  Testing  -  First  subsection  of  three  students  began  testing  Thursday  morning. 
Each  student  had  to  solve  three  separate  problems,  one  each  on  three  different  AN/WSC- 
3  UHF  transceivers.  The  second  subsection  began  testing  Thursday  afternoon,  addressing 
the  same  three  problems.  Both  subsections  concluded  testing  Friday,  addressing  the  final 
two  problems  (again,  with  a  morning  and  an  afternoon  subsection  of  students). 

•  C-School  Critique  -  The  modified  C-School  Critique  was  individually  administered  to 
each  student  on  Friday,  immediately  following  completion  of  the  final  hands-on  test 
problem. 


TABLE  D-3.  SCHEDULE  OF  CBT  REMOTE  GROUP  TRAINING  &  TESTING 
ACTIVITIES  _ 


Monday  Tuesday 


Introduction  CBT  Course 

(1/2  hr.) 

Background  Ques.  (Continue) 
Pre-test 

CBT  Course  Start 


CBT  Course 


(Continue) 


Testing  Intro.(l/2  hr) 
Lab  Familiarization 


Hands-on  T  ests  Hands-on  T  ests 

(AN/WSC-3  Lab)  (AN/WSC-3  Lab) 
Problems  1  -3  Problems  4,5 

Subsection  A  Subsection  B 


C-School  Critique 


CBT  Course 
(Continue) 


CBT  Course 
(Continue) 


Hands-on  training 


Hands-on  Tests  Hands-on  Tests 
(AN/WSC-3  Lab)  (AN/WSC-3  Lab) 
Problems  1  -  3  Problems  4,  5 

Subsection  B  Subsection  A 


Students 

Arrive 


C-School  Critique 


D.3  RESIDENT  GROUP  PROCEDURES 

The  RS  students  reported  to  TRACEN  Petaluma  on  the  day  before  the  start  of  training  and 
experiment  activities,  similar  to  the  RM  students  (i.e.,  Sunday),  and  in  accordance  with  normal 
practice  for  the  traditional  resident  course.  The  students  began  training  Monday  morning  in 
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accordance  with  the  usual  practice  for  the  AN/WSC-3  resident  course.  The  instructor  started  the 
course  with  an  introduction  to  the  experiment,  and  administered  the  Student  Background 
Questionnaire  and  Pretest.  After  that  point,  and  until  the  conclusion  of  training  activities  Friday, 
the  AN/WSC-3  course  was  conducted  in  accordance  with  normal  practice,  with  no  deviations  for 
experimental  purposes. 

The  post-training  and  testing  process  began  Friday  afternoon,  and  concluded  during  Monday  and 
Tuesday  of  the  following  week.  This  testing  process  was  identical  to  that  of  the  DS  and  RS 
groups,  with  the  exception  of  the  time  schedule,  which  differed  for  each  group. 

Instructor  Tasks.  The  instructor  conducted  training  in  accordance  with  normal  practices  for  the 
resident  AN/WSC-3  course.  The  only  instructor  tasks  that  departed  from  normal  practice  were 
those  before  the  start  of  the  training,  namely  the  introduction  and  administration  of  the  Student 
Background  Questionnaire  and  the  Pretest.  Guidance  material  was  provided  to  the  instructor, 
including  a  guide  and  an  experiment  introduction  outline  (For  the  first  day’s  student 
introduction). 

TRACEN  Petaluma  Procedures  (Pre-training,  training,  post-training  and  testing).  All  RS  group 
activities  were  conducted  at  TRACEN.  The  procedures  associated  with  the  pre-training, 
traditional  AN/WSC-3  course,  post-training  and  testing  activities,  were  (See  the  training  and 
testing  schedule  in  Table  D-4): 

•  Introduction  -  Performed  by  the  instructor  Monday  morning,  addressing  experiment  and 
training  activities. 

•  Student  Background  Questionnaire  -  Administered  by  the  instructor,  Monday  morning. 

•  Pretest  -  Administered  by  the  instructor,  Monday  morning. 

•  AN/WSC-3  Course  (Traditional)  -  Started  Monday  morning  at  0930,  and  continued  until 
Friday  afternoon  at  1400.  The  typical  resident  AN/WSC-3  courses  conclude  by  this  time 
on  Friday.  The  instructor  conducted  the  course  in  accordance  with  normal  practices. 

•  Testing  Introduction  -  Performed  by  the  evaluators  Friday  afternoon  at  1430,  addressing 
the  testing  schedule. 

•  Laboratory  Familiarization  -  Conducted  during  a  2-hour  period  Friday  afternoon. 

•  Posttest  -  Administration  of  the  Posttest  Friday  afternoon  (Approximately  1  hour). 

•  Hands-On  Testing  -  First  subsection  of  three  students  began  testing  Monday  morning. 
Each  student  had  to  solve  three  separate  problems,  one  each  on  three  different  AN/WSC- 
3  transceivers.  The  second  subsection  began  testing  Monday  afternoon,  addressing  the 
same  three  problems.  Both  subsections  concluded  testing  Tuesday,  addressing  the  final 
two  problems  (again,  with  a  morning  and  an  afternoon  subsection  of  students). 

•  C-School  Critique  -  The  modified  C-School  Critique  was  individually  administered  to 
each  student  on  Tuesday,  immediately  following  completion  of  the  final  hands-on  test 
problem. 


TABLE  D-4.  SCHEDULE  OF  RESIDENT  GROUP  TRAINING  &  TESTING 


ACTIVITIES 


Time 

Monday 

EiiTTTT  i 

Friday 

Saturday 
&  Sunday 

Monday 

Tuesday 

0730 

Introduction 
(1/2  hr.) 

Course 

(Continue) 

Course  (Continue) 

Hands-on  Tests 

Hands-on  Tests 

0800 

Background  Ques. 
Pre-test 

(AN/WSC-3  Lab) 
Problems  1  -3 

(AN/WSC-3  Lab) 
Problems  4,5 

0900 

Subsection  A 

Subsection  B 

Course  Start 

1000 

1100 

C-School  Critique 

1200 

1300 

Course  (Continue) 

Course 

(Continue) 

Course 

Completion 

"  •  ’ 

Hands-on  Tests 
(ANAVSC-3  Lab) 

Hands-on  Tests 
(ANAVSC-3  Lab) 

1400 

Problems  1  -  3 

Problems  4,5 

'Testing  Intro.  & 

Subsection  B 

Subsection  A 

1500 

Lab 

Familiarization 

V;;= 5 

1600 

f.  ..  ’ 

Post-test  (1  hour) 

1700 

C-School  Critique 
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APPENDIX  E 


ANALYSIS  OF  CRITIQUES  AND  QUESTIONNAIRE 

{ 

This  appendix  contains  four  tables: 

■<  Table  E-l  This  table  presents  the  response  frequencies  for  the  CBT  Duty  Station  Group 

(DS),  to  all  Likert-type  questions  in  their  version  of  the  C-School  Critique.  There 
were  seventeen  students  in  this  group. 

Table  E-2  This  table  presents  the  response  frequencies  for  the  CBT  Remote  Group  (RM),  to 
all  Likert-type  questions  in  their  version  of  the  C-School  Critique.  There  were 
sixteen  students  in  this  group. 

Table  E-3  This  table  presents  the  response  frequencies  for  the  Resident  Group  (RS),  to  all 
Likert-type  questions  in  their  version  of  the  C-School  Critique.  There  were 
thirteen  students  in  this  group. 

Table  E-4  This  table  presents  the  response  frequencies  of  the  duty  station  facilitators  to  all 
Likert-type  questions  in  the  Facilitator  Questionnaire.  Fourteen  facilitators  filled 
out  the  questionnaire  and  returned  it  to  the  R&DC  Project  Officer. 
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TABLE  E-l.  CBT  DUTY  STATION  GROUP  RESPONSES  TO  C-SCHOOL  CRITIQUE 


Questions 

N/A 

Question 

(Frequent 

Strongly 

Disagree 

Response  C 
r of  Student 

Disagree 

ategories 

Responses) 

Agree 

Strongly 

Agree 

1 .  Jam  very  confident  of  my  ability  to  maintain  this  equipment. 

0 

0 

0 

12 

5 

2.  Testing  clearly  measured  my  ability  to  perform  each  objective. 

0 

0 

2 

12 

3 

3.  Plenty  of  practice  is  provided  for  each  objective. 

0 

1 

10 

6 

0 

4.  Proper  troubleshooting  techniques  are  taught,  and  tested. 

0 

0 

2 

13 

2 

5 .  Troubleshooting  problems  are  realistic. 

0 

0 

0 

14 

3 

6.  Course  is  presented  in  a  logical  sequence. 

0 

0 

4 

12 

0 

7.  Course  length  is  ideal.  ** 

0 

0 

4 

12 

0 

8.  Proper  tools  and  test  equipment  are  always  available  in  the  computer 
media. 

1 

1 

1 

12 

2 

9.  Spare  parts  are  available  and  ready  for  use  in  computer  media.** 

1 

0 

1 

11 

2 

10.  Safety  equipment  is  always  available.  ** 

1 

0 

1 

11 

3 

1 1 .  Training  and  visual  aids  enhanced  learning. 

0 

0 

1 

11 

5 

12  Technical  manuals  are  current,  legible,  and  accessible. 

0 

0 

1 

11 

5 

13.  HELP  information  on  the  computer  was  understandable  and  enhanced 
training. 

0 

0 

0 

12 

5 

1 4.  Course  work  directly  relates  to  course  objectives. 

0 

0 

0 

15 

2 

15.  Welcome  aboard  package  is  very  helpful. 

6 

0 

3 

8 

1 6.  The  facilitator  clearly  and  concisely  answered  all  questions. 

0 

1 

7 

7 

2 

1 7.  Facilitator  maintained  control. 

0 

0 

7 

8 

2 

1 8.  Facilitator  is  knowledgeable,  skillful,  and  confident  in  assisting  training 
process. 

0 

0 

6 

8 

3 

1 9.  Facilitator  strictly  enforced  safety  standards . 

3 

0 

4 

7 

3 

20.  Training  environment  was  comfortable  and  functional. 

1 

2 

2 

9 

3 

2 1 .  There  is  good  visibility  in  training  room/spaces. 

0 

0 

3 

10 

4 

22.  There  were  no  distractions  during  training. 

0 

2 

5 

7 

3 

23.  Training  time  was  never  wasted. 

1 

1 

3 

9 

3 

24.  I  was  well  prepared  to  take  this  AN/WSC-3  course. 

1  ; 

1 

2 

12 

1 

25.  The  AN/WSC-3  course  was  directly  relevant  to  my  job. 

1 

0 

1 

13 

2 

26.  The  quality  of  training  was  satisfactory. 

0 

0 

3 

11 

3 

27.  The  course  was  effective  overall.  ** 

0 

0 

0 

13 

3 

28.  The  AN/WSC-3  computer-based  course  was  as  effective  as  a  traditional 
classroom/lab  course. 

0 

1 

9 

5 

2 

29.  The  level  of  AN/WSC-3  instruction  detail  was  sufficient.  ** 

0 

0 

4 

11 

1 

30.  The  AN/WSC-3  computer-based  course  was  challenging. 

0 

0 

4 

13 

0 

o  ! 

1 

3 

11 

2 

32.  The  computer  medium  was  satisfactory  for  the  practice  problems. 

0 

1 

3 

11 

2 

33.  Some  AN/WSC-3  equipment  hands-on  training  should  be  provided  after 
completing  the  computer-based  part  of  the  course. 

0 

0 

2 

5 

10 

34.  The  computer-based  training  materials  were  user  friendly 

1 

0 

1 

6 

9 

35.  The  computer  and  software  worked  satisfactorily. 

1 

0 

0 

7 

9 

36.  1  had  no  difficulty  learning  to  use  the  computer  for  this  course. 

0 

0 

0 

5 

12 

37.  The  training  environment  (room,  table/desk  where  trained  on  the 
computer)  was  satisfactory. 

0 

0 

5 

8 

4 

38.  The  time  made  available  for  me  to  do  the  course  was  adequate. 

0 

1 

5 

8 

3 

39.  Computer-based  training  at  duty  stations  can  be  effective  in  the  Coast 
Guard. 

1 

0 

1 

8 

7 

40.  My  command  strongly  supported  this  AN/WSC-3  computer-based 
training. 

2 

0 

1 

8 

6 

41.  I  had  no  difficulty  in  setting  up  and  conducting  the  computer-based 
training. 

1 

0 

1 

11 

4 

42.  The  computer-based  training  conducted  at  my  duty  station  was 
convenient. 

0 

1 

6 

6 

4 

43.  There  was  very  little  interference  from  others  during  my  training 
sessions. 

1 

1 

6 

5 

4 

44.  The  computer  and  all  necessary  guidance  and  training  materials  were 
satisfactorily  received  at  my  duty  station. 

0 

9 

8 

Note:  **  Denotes  one  or  two  students  failed  to  answer  question 
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TABLE  E-2.  CBT  REMOTE  GROUP  RESPONSES  TO  C-SCHOOL  CRITIQUE 


Questions 

Question  Response  Categories 
(Frequency  of  Student  Responses) 

N/A 

Strongly 

Disagree 

Disagree 

Agree 

Strongly 

Agree 

1  I  am  very  confident  of  my  ability  to  maintain  this  equipment. 

0 

0 

1 

9 

6 

2  Testing  clearly  measured  my  ability  to  perform  each  objective. 

0 

0 

2 

13 

1 

3  Plentv  of  practice  is  provided  for  each  objective. 

0 

0 

5 

11 

0 

4.  Proper  troubleshooting  techniques  are  taught,  and  tested. _ 

0 

0 

1 

8 

7 

5.  Troubleshooting  problems  are  realistic. 

0 

0 

0 

10 

6 

6.  Course  is  presented  in  a  logical  sequence. 

0 

0 

0 

8 

8 

7.  Course  length  is  ideal. 

0 

1 

4 

7 

4 

8.  Proper  tools  and  test  equipment  are  always  available  in  the  computer 
media. 

0 

0 

0 

10 

6 

9  Snare  parts  are  available  and  ready  for  use  in  computer  media.  ** 

0 

0 

1 

8 

6 

10.  Safety  equipment  is  always  available. 

0 

0 

0 

5 

11 

1 1 .  Training  and  visual  aids  enhanced  learning. 

0 

0 

0 

8 

8 

1 2  Technical  manuals  are  current,  legible,  and  accessible. 

0 

0 

1 

8 

7 

13.  HELP  information  on  computer  is  understandable  and  enhances 
training. 

0 

0 

0 

9 

7 

1 4.  Course  work  directly  relates  to  course  objectives. 

0 

0 

0 

11 

5 

15.  Welcome  aboard  package  is  very  helpful. 

7 

0 

1 

8 

0 

1 6  The  facilitator  clearly  and  concisely  answered  all  questions. 

0 

0 

3 

7 

6 

1 7.  Facilitator  maintained  control. 

o  ~i 

0 

0 

5 

11 

18.  Facilitator  is  knowledgeable,  skillful,  and  confident  in  assisting 
training  process. 

2 

0 

1 

6 

7 

1 9  Facilitator  strictly  enforced  safety  standards. 

2 

0 

0 

7 

7 

20  Training  environment  was  comfortable  and  functional. 

0 

0 

0 

10 

6 

21  There  is  good  visibility  in  training  room/spaces. 

0 

0 

0 

9 

7 

22.  There  were  no  distractions  during  training. 

0 

0 

0 

9 

7 

23.  Training  time  was  never  wasted. 

0 

0 

4 

7 

5 

24.  I  was  well  prepared  to  take  this  AN/WSC-3  course. 

0 

0 

3 

11 

2 

25.  The  AN/WSC-3  course  was  directly  relevant  to  my  job. 

0 

0 

0 

12 

4 

26.  The  quality  of  training  was  satisfactory. 

0 

0 

0 

10 

6 

27.  The  course  was  effective  overall. 

0 

0 

0 

10 

6 

28.  The  AN/WSC-3  computer-based  course  was  as  effective  as  a 
traditional  classroom/lab  course. 

0 

1 

0 

10 

5 

29  The  level  of  AN/WSC-3  instruction  detail  was  sufficient.  ** 

0 

0 

3 

11 

1 

30  The  AN/WSC-3  computer-based  course  was  challenging. 

0 

0 

3 

11 

2 

3 1  The  amount  of  general  theory  provided  was  sufficient. 

1 

0 

2 

10 

3 

32  The  computer  medium  was  satisfactory  for  the  practice  problems. 

0 

0 

2 

11 

3 

33.  Some  AN/WSC-3  equipment  hands-on  training  should  be  provided 
after  completing  the  computer-based  part  of  the  course. 

0 

0 

2 

2 

12 

34.  The  computer-based  training  materials  were  user  friendly  and  easy  to 
use. 

0 

0 

0 

7 

9 

35 .  The  computer  and  software  worked  satisfactorily. 

0 

0 

0 

8 

8 

36  I  had  no  difficulty  learning  to  use  the  computer  for  this  course. 

0 

0 

0 

8 

8 

37.  The  training  environment  (room,  table/desk  where  trained  on  the 
computer)  was  satisfactory. 

0 

0 

0 

8 

8 

38  The  time  made  available  for  me  to  do  the  course  was  adequate. 

0 

0 

0 

10 

6 

39.  Computer-based  training  at  duty  stations  can  be  effective  in  the 
Coast  Guard. 

0 

0 

3 

8 

5 

Note:  **  Denotes  one  or  two  students  failed  to  answer  question 


TABLE  E-3.  RESIDENT  GROUP  RESPONSES  TO  C-SCHOOL  CRITIQUE 

I  Question  Response  Categories 

(Frequency  of  Student  Responses) 


N/A  Strongly  Disagree 
Disagree 


Agree  Strongly 
Agree 


1 .  lam  very  confident  of  my  ability  to  maintain  this  equipment. 

2.  Testing  clearly  measured  my  ability  to  perform  each  objective. 


Plenty  of  practice  is  provided  for  each  objective.  ** 


Proper  troubleshooting  techniques  are  taught,  and  tested.  ** 


Troubleshooting  problems  are  realistic. 


j.  Course  is  presented  in  a  logical  sequence. 


7.  Course  length  is  ideal.  _ 


Proper  tools  and  calibrated  test  equipment  are  always  available. 


Spare  parts  are  plentiful  and  ready  for  issue. 


10.  Safety  equipment  is  always  available. 


1 1 .  Training  and  visual  aids  enhanced  leamin 


1 2  Technical  manuals  are  current,  legible,  and  accessible. 


1 3.  Handouts  are  legible  and  enhance  trainin 


14.  Course  work  directly  relates  to  course  obiectives. 


15.  Welcome  aboard  package  is  very  helpful. 


1 6.  The  instructor  clearly  and  concisely  answered  all  questions. 


1 7.  Instructor  maintained  interest  and  control  of  class. 


18.  Instructor  is  knowledgeable,  skillful,  and  confident. _ 


1 9.  Instructor  strictly  enforced  safety  standards. 


20.  Classroom  and  labs  are  comfortable,  and  functional.  ** _ 


2 1 .  There  is  good  visibility  in  classroom  and  lab. 


22.  There  are  no  distractions  to  the  class. 


23.  Class  time  is  never  wasted.  ** 


24.  I  was  well  prepared  to  take  this  AN/WSC-3  course.  ** 


25.  The  AN/WSC-3  course  was  directly  relevant  to  my  job.  ** 


26.  The  quality  of  training  was  satisfacto 


27.  The  course  was  effective  overall. 


28.  The  level  of  AN/WSC-3  instruction  detail  was  sufficient. 


I  29.  The  amount  of  general  theory  provided  was  sufficient. _ 


Note:  **  Denotes  one  or  two  students  failed  to  answer  question 


TABLE  E-4.  FACILITATOR  RESPONSES  TO  FACILITATOR  QUESTIONNAIRE 

- —  “  "  Question  Response  Categories 

Questions  (Frequency  of  Facilitator  Responses) 


1 .  The  duty  station  command  strongly  supported  this  trainin _ 

2.  The  computer,  headphones,  and  guidance  and  training  materials  were 

satisfactorily  received  at  my  duty  station. _ _ _ 

3.  All  materials  and  information  needed  to  complete  the  training  were 
_ provided. 

4.  The  guidance  information  for  facilitating  the  computer-based 

_ training  was  satisfactory.  _ _ _ _ 

5 .  The  training  facilitation  task  was  performed  without  difficulty. _ 

6.  Scheduling  of  training  sessions  was  not  a  problem. 

7.  Finding  appropriate  space  for  the  student  to  use  during  training  was 

not  a  problem. _ _ _ _ _ — 

8  The  student’s  training  sessions  ran  smoothly. _ _ 

9.  There  was  little  interference  from  others  during  the  training  sessions. 

1 0.  Interaction  with  the  Student  was  satisfactory.  _ 

1 1 .  The  self-paced  method  of  this  course  was  convenient. _ 

1 2  Overall  quality  of  the  computer-based  media  was  satisfactory. _ 

1 3 .  The  computer  and  software  worked  satisfactorily.  _ 

1 4.  Computer-based  training  courses,  similar  to  this  one,  can  be  effective 

at  duty  stations.  _ _ _ 

15.  Student  travel  arrangements  were  handled  satisfactorily. _ 


Strongly  Disagree 
Disagree  _ _ 


Strongly 

Agree 


E-5 


